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What Is a TIP? 


CSAT Treatment Improvement Protocols (TIPs) are prepared by the Quality Assurance and 
evaluation Branch to facilitate the transfer of state-of-the-art protocols and guidelines for the 
treatment of alcohol and other drug (AOD) abuse from acknowledged clinical, research, and 
administrative experts to the Nation's AOD abuse treatment resources. 

The dissemination of a TIP is the last step in a process that begins with the recommendation of 
an AOD abuse problem area for consideration by a panel of experts. These include clinicians, 
researchers, and program managers, as well as professionals in such related fields as social 
services or criminal justice. 

Once a topic has been selected, CSAT creates a Federal resource panel, with members from 
pertinent Federal agencies and national organizations, to review the state of the art in treatment 
and program management in the area selected. Recommendations from this Federal panel are 
then communicated to the members of a second group, which consists of non-Federal experts 
who are intimately familiar with the topic. This group, known as a non-Federal consensus panel, 
meets in Washington for 5 days, makes recommendations, defines protocols, and arrives at 
agreement on protocols. Its members represent AOD abuse treatment programs, hospitals, 
community health centers, counseling programs, criminal justice and child welfare agencies, and 
private practitioners. A Chair for the panel is charged with responsibility for ensuring that the 
resulting protocol reflects true group consensus. 

The next step is a review of the proposed guidelines and protocol by a third group whose 
members serve as expert field reviewers. Once their recommendations and responses have been 
reviewed, the Chair approves the document for publication. The result is a TIP reflecting the 
actual state of the art of AOD abuse treatment used in public and private programs recognized 
for their provision of high quality and innovative AOD abuse treatment. 

Outcomes monitoring can be defined as the assessment, at some goal point following treatment, 
of patient progress toward the program’s goals of treatment. Outcomes montioring can help 
determine the efficacy of treatment for different types of patients, improve program performance, 
and document cost-effectiveness. In addition, the data derived from outcomes monitoring 
systems(OMSs) can be used to make resource allocations, justify funding requests, and provide 
data on outcomes to managed care systems. 

This TIP, entitled Developing State Outcomes Monitoring Systems for Alcohol and Other Drug 
Abuse Treatment Programs, can be of help to anyone interested in improving patient matching 
and treatment in this era of increasing emphasis on outcomes accountability. 

This TIP represents another step by CSAT toward its goal of bringing national leadership to bear 
in the effort to improve AOD abuse treatment. 



Other TIPs may be ordered by contacting the National Clearinghouse for Alcohol and Drus 
Information (NCADI) 800-729-6686 or (301) 468-2600; TDD (for hearing impaired), (800) 487- 
4889. 
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Foreword 


The Treatment Improvement Protocol Series (TIPs) fulfills CSAT's mission to improve alcohol 
and other drug (AOD) abuse and dependency treatment by providing best practices guidance to 
clinicians, program administrators, and payers. This guidance, in the form of a protocol, results 
from a careful consideration of all relevant clinical and health services research findings, 
demonstration experience, and implementation requirements. A panel of non-Federal clinical 
researchers, clinicians, program administrators, and patient advocates employs a consensus 
process to produce the product. This panel’s work is reviewed and critiqued by field reviewers as 
it evolves. 

The talent, dedication, and hard work that TIPs panelists and reviewers bring to this highly 
participatory process have bridged the gap between the promise of research and the needs of 
practicing clinicians and administrators. I am grateful to all who have joined with us to 
contribute to advance our substance abuse treatment field. 

David J. Mactas 
Director 

Center for Substance Abuse Treatment 



Chapter 1—Introduction 

Outcomes monitoring systems (OMSs) are broad-based efforts that aggregate data from many 
alcohol and other drug (AOD) abuse treatment programs. Their goal is to assess and ultimately 
improve patient outcomes, including the individual's functioning in society following treatment. 
An OMS can help to establish accountability for the expenditure of public funds. To allow the 
State to address the relevant questions, the OMS must, at the minimum, include sufficient 
information on patients, their treatment and their posttreatment functioning. 

In each of the United States and territories, a State agency has responsibility for ensuring that 
AOD abuse treatment programs receiving Federal dollars provide effective treatment at the 
lowest possible cost. This assurance is difficult to give because many different approaches to 
providing treatment are employed today across the country, and patients have many different 
types of substance use disorders. The burden on the State agency to ensure accountability has 
intensified in recent years as a result of the rising costs for provision of healthcare services and 
the healthcare reform movement. 

The healthcare reform movement has brought about changes in the management of healthcare in 
numerous States, leading many single State agencies (SSAs) to recognize the need for initiating 
outcomes monitoring or augmenting existing efforts. SSA directors and staff, as well as involved 
constituencies, need assistance in developing and improving OMSs. This Treatment 
Improvement Protocol (TIP) provides needed information and guidelines. 


The burden on the State agency to ensure accountability has intensified in recent years. 


Purposes and Audiences of This TIP 

As part of its ongoing commitment to identify and respond to critical issues in the AOD field, the 
Center for Substance Abuse Treatment (CSAT) convened a panel of professionals in AOD abuse 
treatment and allied fields. The panel was charged with examining outcomes-based 
accountability as it applies to AOD treatment and with conceptualizing and developing a TIP on 
the subject. In preparing this TIP, the panel was asked to do the following: 

• Consider what monitoring activities currently exist 

• Identify supportive resources and obstacles to improving the information base 

• Make recommendations to SSAs about the creation and operation of outcomes monitoring 
systems. 








Thus, the purpose of this TIP is to assist single State agencies in the development, 
implementation, and management of an OMS to improve treatment outcomes and to increase 
accountability for AOD treatment expenditures. 

Because the success of a State OMS will depend on collaboration among many stakeholders, 
there are a number of agencies and professionals in the audience for this TIP: 

• AOD abuse treatment planners, administrators, and research and evaluation personnel at State 
AOD agencies who are responsible for the design, implementation, management, or oversight of 
treatment programs 

• Utilization reviewers, quality assurance personnel, and staff of managed care programs who 
evaluate the outcomes of AOD treatment programs 

• AOD direct service providers including physicians, psychologists, counselors, social workers, 
nurses, and therapists who provide the full range of AOD services 

• Other interested allied health professionals 

• Professionals from other fields who encounter the consequences of AOD abuse, such as criminal 
justice, public health, and social services 

• Third-party payers who have a stake in ensuring not only the effectiveness of AOD abuse 
treatment but also its cost-effectiveness. 


Forces Leading to the Development of This TIP 

The field of AOD abuse treatment has not been exempt from the influence of forces that have 
been acting on the Nation's healthcare delivery system. The Nation has moved into an era of 
increasing emphasis on outcomes-based accountability (Reiman, 1988). While calls for 
healthcare reform emphasize universal coverage and cost containment, access to care and low- 
cost care alone will not guarantee improved healthcare. Reform efforts must require the use of 
effective forms of treatment that have demonstrated their value in curing or arresting disease or 
relieving patients’ distress. Decisions about the best care for individual patients must be based on 
outcomes (Alio et ah, 1988; Longabaugh, 1991). 

Alcohol and other drug abuse treatment has always received intensive scrutiny—far more than 
have other healthcare services. This scrutiny is probably because of the nature of addiction and 
the visibility of its effects. Although attitudes have changed greatly over recent decades, persons 
with AOD problems are still viewed with a great deal of ambivalence. For example, many people 
who say that alcoholism is an illness also say that alcoholics drink because they want to and that 
alcoholics are morally weak individuals (Caetano, 1989; Gallup, 1987). Abuse of illegal drugs 
elicits even harsher judgments. Media attention to so-called treatment failures that involve 
repeated offenses of driving while intoxicated or other more highly publicized crimes increases 
public skepticism about the benefits of treatment. 

Despite lingering reservations about treatment effectiveness, a considerable body of research 
documents the fact that receiving AOD treatment is better than not receiving it and that treatment 
costs are offset by savings in other areas (Luckey, 1987). Treatment leads to substantial 


reductions in alcohol and drug problems; improvements in virtually all other areas of patient 
functioning, including physical health, psychological and social functioning, and employment; 
and reduced incidence of criminal behavior (Hubbard et ah, 1989; McLellan et ah, 1992a; 
Pickens and Fletcher, 1991; Tims et ah, 1991). Cost-offset studies consistently document 
reduced healthcare costs following treatment for AOD abuse (Holder, 1987; Holder and Blose, 
1992, 1986; Holder and Hallan, 1986, 1981; Holder and Shachtman, 1987; Holder et al, 1985; 
Jones and Vischi, 1979), along with reduced criminality and increased employee productivity 
(Alander and Campbell, 1975; Alfano et al., 1987; Harrison and Hoffmann, 1989; Hoffmann et 
al., 1984; Hubbard et al., 1989). Findings from State studies are remarkably consistent in this 
regard (Young, 1994). In the most rigorous and comprehensive study of cost offsets conducted to 
date, a return on taxpayer investment of $7.14 for every $ 1 spent on AOD treatment was recently 
documented in California (California Department of Alcohol and Drug Programs, 1994). 

However, while it is known that treatment is effective, it is also known that no single treatment 
approach is effective for all persons with AOD problems. Outcomes are determined by a number 
of factors: the characteristics of individuals seeking treatment, the nature and severity of their 
problems, the treatment process and the services provided, posttreatment environmental 
conditions, and the interactions among these factors (Ball and Ross, 1991; Bromet and Moos, 
1977; Budde et al., 1992; Berglund et al., 1991; French et al., 1993; Gerstein and Harwood, 

1990; Harrison et al., 1988; Institute of Medicine, 1990; Joe et al., 1992; McLellan et al., 1993, 
Vanicelli et al., 1983). 


Several Factors Affecting Outcomes 

• The characteristics of individuals seeking treatment 

• The nature and severity of their problems 

• The treatment process and the services provided 

• Posttreatment environmental conditions 

• The interactions among these factors. 


Potential Impact of OMSs on The Treatment of Substance Use Disorders 

Much more about treatment could be learned from well-planned outcomes monitoring systems. 
Such systems could provide answers to many questions: What kind of treatment is best for 
different populations of patients? What specific components of treatment are essential to 
recovery? What are the relationships between the specific treatment services patients receive and 
the outcomes of their treatment? Do different kinds of treatment settings provide benefits to 
different kinds of patients? Do particular combinations of services improve outcomes? Does 








increasing or decreasing the length, the frequency, or the intensity of specific services improve 
prognosis? To what extent does the context of treatment influence outcomes? Although some 
programs offer gender-specific services or services tailored to cultural or ethnic populations, 
selected age groups, or other special populations, we do not yet know whether these specialized 
services have affected treatment outcomes. 

It is widely believed that matching patients to programs or services that best meet their needs 
will improve treatment outcomes (Institute of Medicine, 1990; Smart et ah, 1990-1991). 
Although results of fairly small-scale studies conducted to date are promising (McLellan et ah, 
1983a, 1983b; Schottenfeld et ah, 1992), much larger efforts involving more diverse populations 
and services will be required to determine if patient-treatment matching attains the goal of 
improving treatment outcomes and to calculate the cost-effectiveness of different treatment 
approaches (McLellan and Alterman, 1991). 

Across the country, wide variations exist in the types of care given to patients with AOD 
disorders. Differences exist within and among States in the use of various therapeutic modalities, 
treatment settings, treatment goals, and variety of services offered, as well as in the quantity of 
specific services available. 

Treatment in the United States typically consists of some combination of services such as 
detoxification, medical care, psychological assessment, educational lectures and films, group 
therapy, individual counseling, family counseling, recreational and occupational therapy, 
medication, use of community-based peer support groups such as Alcoholics Anonymous, and 
continuing care or relapse prevention sessions. Depending on the modality, treatment may last 
anywhere from a few sessions to a year or longer. Much information about the benefits of these 
different combinations of services could be learned from outcomes monitoring systems. 


Outcomes Monitoring and The State 

The single State agency has a key role with respect to AOD treatment. Its job is to ensure the 
provision of high-quality care for patients, based on their individual needs, in order to achieve 
the best outcomes for the lowest possible costs. State agencies typically play a role in the 
provision of program funding or provider reimbursements, program development, program 
licensing, and/or patient assessment and placement standards. An outcomes monitoring system, 
as described in this TIP, is a type of management information system (MIS) designed to collect 
sufficient information to allow the State to fulfill its role and responsibilities in these areas. To 
address the relevant questions, the OMS must, at a minimum, include sufficient information on 
patients, the treatment they receive, and how they are doing after treatment. 


An outcomes monitoring system, as described in this TIP, is a type of management information system 
(MIS) designed to collect sufficient information to allow the State to fulfill its role in the provision of 







program funding or provider reimbursements, program development, program licensing, patient 
assessment, and placement standards. 


Outcomes monitoring can be defined as the assessment, at some point following treatment, of 
patient status in key life areas related to substance use disorders. The purposes of outcomes 
monitoring are the following: 

• Help States, treatment professionals, and policymakers determine the effectiveness of certain 
types of AOD abuse treatment for different types of patients 

• Improve program performance by using outcomes data to identify weaknesses or gaps in 
services, and provide feedback to enhance system performance 

• Improve the patient assessment process, using empirical outcomes data to develop and refine 
treatment placement criteria that optimize the chance of successful outcomes 

• Document cost offsets and minimize inefficiencies and unnecessary expenditures in treatment 
programs 

• Help improve resource allocations 

• Provide justification to support funding requests, especially the use of public dollars, and the 
necessity of including treatment in a basic healthcare benefits package 

• Improve managed care by providing data on treatment outcomes to guide decisions about the 
wise use of the healthcare dollar. 


Relationship of Outcomes Monitoring Systems to Quality Improvement 

Outcomes monitoring systems share characteristics in common with experimental research, 
quality assurance and improvement efforts, and program evaluation. All include data gathering 
and, to varying degrees, require some standardization of instruments and procedures and training 
of data-gathering personnel. However, there are differences in outcomes monitoring systems, 
particularly in their purposes and scopes. 

Experimental Research 

Experimental research has as its primary goal the accumulation of knowledge and thus 
emphasizes theory development and hypothesis testing. Experimental designs involve the 
randomization of subjects and the use of control groups. Only experimental designs can establish 
a causal link between a specific intervention and a treatment outcome. Most experimental 
research is relatively expensive because it involves comprehensive data collection by extensively 
trained staff and the greatest degree of technical sophistication in the outcome evaluation 
approaches reviewed here. 

As an example, consider a project designed to test a new medication to determine whether it 
helps to alleviate the subjective experience of craving among persons addicted to cocaine. The 
project design might include six outpatient treatment sites, each recruiting 60 volunteers. Patients 
would be told that the research involves the use of an experimental medication and random 







assignment to one of three groups: a group receiving the experimental drug, a group receiving an 
antidepressant medication proven to help reduce craving, and a group receiving a placebo. The 
pills each group received would look alike; neither the patients nor the doctors and staff involved 
in the study would know which patient was receiving which medication (this is called a double¬ 
blind study). 

Patients in all three groups would also receive the standard treatment provided by the outpatient 
program to which they were admitted. For 12 weeks, the patients would meet with staff once a 
week to complete questionnaires on the frequency and intensity of craving. Staff would also 
record their observations of patient behavior. At the end of the study, the phannaceutical 
company testing the medication would analyze the data on the three groups of patients. 

Even if the findings showed an effect of the experimental drug on alleviating a craving for 
cocaine, changes in clinical practice are dependent on a variety of outside factors. These include 
approval by the U.S. Food and Drug Administration, the price of the drug, the availability of 
other medications, loyalty of clinical practitioners to other drugs, or even philosophical 
objections to the use for this purpose of any medication for people with addictions. Thus, the 
knowledge gained as a result of experimental research may be far removed from changes in 
clinical practice or general healthcare policy. 

Quality Assurance and Improvement 

Historically, quality assurance efforts have been focused on the delivery of services, not 
outcomes. There is now greater attention on outcomes as well. Two well-known efforts in this 
area are conducted by the Joint Commission on Accreditation of Healthcare Organizations 
(JCAHO) and the Commission on Accreditation of Rehabilitation Facilities (CARF). Licensure 
agencies provide other examples of quality assurance. 

Since 1992, JCAHO has been revising its standards on quality assessment and improvement to 
reflect increasing emphasis on effective and efficient use of healthcare resources (JCAHO, 

1992). JCAHO and similar organizations, in collaboration with healthcare professionals, develop 
quality standards and provide assistance to healthcare providers in meeting or exceeding those 
standards. Such efforts typically examine the full spectrum of governance, management 
practices, and clinical processes. They focus on structures and activities and the integration and 
coordination of these related aspects of program performance. The goal is continual quality 
improvement. 

What distinguishes quality assurance efforts of this type from individual program evaluation is 
their broad-based nature. Through consensus, unifonn standards are adopted and individual 
program performance is assessed against these practice parameters. Accreditation or licensure by 
an external reviewer attests to the achievement of established standards. 

Program Evaluation 

As with quality assurance, the purpose of program evaluation is improved service delivery. 
Program evaluation designs, however, are typically individualized to meet the unique needs of a 



program or group of programs (see accompanying box). The evaluation may be initiated by the 
program itself or it may be mandated by an external funding or review agency. The audience 
includes program administrators and clinical staff as well as funders. Like quality assurance, 
program evaluation may involve direct observations of service delivery, review of records, and 
interviews with patients, staff, and other stakeholders. Data collected may be qualitative as well 
as quantitative. The emphasis is on program performance, not individual patient outcomes. 
Program evaluation has recently evolved to include more of an emphasis on patient outcomes, 
but the collection of outcomes data is often not systematic. 


Summary Description of Outcomes Monitoring Systems 

Outcomes monitoring systems are broad-based efforts that aggregate data from many programs. 
The primary goal of outcomes monitoring systems is to assess and ultimately improve patient 
outcomes. An OMS can help to establish accountability for the expenditure of public funds. 
Examples of existing State outcomes monitoring systems are provided in Appendix B. 

While there may be many ways to use an OMS, the primary audience is broader than the 
participating programs and may include State or Federal regulatory agencies or other groups with 
a stake in the treatment's being monitored. Findings from outcomes monitoring systems may lead 
to policy changes at the State or Federal levels or by other healthcare payers. 

Outcomes monitoring systems use standardized data elements and data collection procedures at 
different sites. OMSs include some contact with the patient or the use of collateral records 
following treatment. OMSs have some components in common both with experimental research 
and with program evaluation, but there are also major differences. The patient—not the 
program—is the unit of data analysis. Monitoring is ongoing rather than time limited. The 
number of patients involved is usually quite large. The system may include a variety of 
treatments in a variety of settings at multiple sites. 

The outcomes monitoring system assesses the treatment delivery system "as is"; patients are not 
involved in experimental research. Because an outcomes monitoring system does not involve the 
use of experimental and control groups, the data collected cannot causally link a specific 
outcome to a specific intervention. However, meaningful treatment service comparisons can be 
achieved because collecting data on the same patients before and after treatment allows each 
patient to serve as his or her own control (California Department of Alcohol and Other Drug 
Programs, 1994). 


Uses of an OMS 

One fundamental question must be resolved when planning an OMS: Is the major objective to 
address questions related to the treatment service delivery system as a whole or to compare 


individual treatment program results? Consensus on this issue may not be easy to achieve. Strong 
arguments can be made for both approaches. 

Advocates who favor designing the OMS to measure individual program performance point to 
the benefits for consumers and payers of identifying superior and inferior programs. Opponents 
of this approach express serious doubts about its feasibility—considering sample sizes necessary 
for valid comparisons and the enormous amount of heterogeneity among treatment populations. 


Hypothetical Example: Program Evaluation 

Consider the example of a new residential treatment program for women with young children, which is 
required, as a condition for continued funding, to conduct a program evaluation. The evaluation may 
include assessment of access to the program, through such means as a survey to determine the level of 
awareness of the new program among community agencies, and their staff's attitudes toward the 
program. The evaluation might include separate interviews or focus groups with participants and staff to 
assess their satisfaction or dissatisfaction with the program in general and specific aspects of the 
services. The evaluation report would describe the program implementation, discuss whether the 
implementation went as planned, and note any changes that were made. It would include quantitative 
summaries of persons served, services provided, and program completion rates. It would also include a 
qualitative analysis of personal reactions gleaned from questionnaires, interviews, or focus groups. The 
report would assess progress to date toward achieving program goals and would include 
recommendations for improving services. 


Measuring treatment system outcomes versus measuring individual treatment program outcomes 
is not necessarily an either/or situation. In any OMS designed to address program results, data 
could also be aggregated to address questions pertaining to the broader treatment system. The 
converse is not necessarily true, however. An OMS could be designed to address questions about 
the treatment system but be insufficient to provide individual program comparisons. 

Members of the national consensus panel for this TIP did not reach consensus on this issue. 
Therefore, the recommendations which follow take into account both approaches. While 
planners of future outcomes monitoring systems may find debate on this issue to be contentious, 
they will also find it worthwhile. The ensuing discussions will bring to the fore many related 
issues: the role of the State in monitoring and improving treatment outcomes; factors involved at 
the program level that may relate to outcomes; the degree of scientific rigor necessary to ensure 
confidence in results; the importance of study design, sample size, and technical expertise in 
making sure the monitoring system can do what it is expected to do; and the benefits and 
drawbacks of linking funding to performance. 

While an OMS can be designed to measure the performance of an individual program, there are 
considerable obstacles to this use of the system. To conduct valid comparisons of treatment 
programs in terms of treatment outcomes, program samples must be sufficiently large to ensure 







that observed differences are statistically significant. Even when outcome differences are 
significant, the poorer outcomes may be attributable to patient characteristics at admission. Many 
patient characteristics are associated with poorer outcomes. These characteristics include severity 
of AOD use, early age of onset, a history of antisocial behavior, vocational and social instability, 
and the severity of psychiatric disorders (Berglund et al., 1991; Budde et ah, 1992; Gottheil et 
ah, 1992; Harrison and Hoffmann, 1987; Harrison et al., 1988, 1991; Longabaugh, 1991; 
McLellan et al., 1983a). 

Two programs may offer comparable quality services by equally skilled staff, but the one that 
serves higher risk patients will probably have poorer outcomes. Unless differences in patient 
characteristics can be controlled in statistical analyses, the outcomes comparisons will not be 
meaningful. Sufficiently large sample sizes can make valid program comparisons possible; 
however, sample size is a key factor in driving up the costs of an OMS. Finally, other factors 
such as sample bias threaten the validity of program comparisons (Gerson et al., 1985; Harrison 
and Hoffmann, 1989; Stinchfield et al., 1994a). An example of the effects of sample bias 
introduced by disparity in consent and posttreatment followup rates is described on the next 
page. 


Discussions on outcomes monitoring systems will bring to the fore the following issues: 

• The role of the State in monitoring and improving treatment outcomes 

• Factors involved at the program level that may relate to outcomes 

• The degree of scientific rigor necessary to ensure confidence in results 

• The importance of study design, sample size, and technical expertise in making sure the 
monitoring system can do what it is expected to do 

• The benefits and drawbacks of linking funding to performance. 


Focusing on the aggregated data from all programs may produce even more important benefits 
than individual program outcome comparisons. An aggregated data set of sufficient size can 
address a variety of broad questions about the correlates of successful outcomes and the best use 
of available resources. In contrast to the report-card approach of measuring individual program 
performance, the systems approach facilitates a self-correcting treatment service delivery system 
in which continuous feedback is used to improve services, make management decisions, and 
refine policy. 

Because an OMS will necessarily include data on patient characteristics, severity of functioning, 
and services provided, along with patient outcomes data, groups of patients can be compared 
with other groups of patients with similar characteristics who receive different services, or 
services that are different in intensity or duration. Pooling data from a diversity of patients and 
treatment models and services will ultimately enhance the ability to determine what kinds of 
service and levels of service are most likely to benefit specific populations of patients. The 
emphasis here is on treatment components (or service packages) rather than on treatment 
programs. Analysis targets the sets of services that comprise treatment rather than the individual 
treatment program. 







Example of Sample Bias 

Suppose you were to compare the abstinence rates 6 months posttreatment for Program A and 
Program B. (Set aside for now whether or not this measure is a good one for program outcomes.) 
Let us even assume that characteristics of the patient populations at both treatment sites were 
virtually identical on all measures that might predict outcomes (again, set aside skepticism). Data 
analysis reveals that Program A has a 50 percent abstinence rate and Program B, a 65 percent 
abstinence rate. Some might want to promote Program B as superior, but closer scrutiny is 
required. 

Both programs had 110 admissions during the study period. At Program A, 100 patients 
consented to followup and 80 of these were contacted for the 6-month followup interview. The 
Program A followup sample consisted of 73 percent (80 /110) of the original admissions; 40 of 
these (50 percent) reported no alcohol or other drug use after treatment. 

At Program B, 95 patients consented to followup and 62 of these were contacted for the 6-month 
followup interview. The followup sample consists of 56 percent (62 / 110) of the original 
admissions; 40 of these reported no alcohol or drug use after treatment. 

The first problem with this finding is that the observed difference in outcomes (40 of 80 
interviewees versus 40 of 62 interviewees reporting abstinence) is not statistically significant. 
That means that there is a greater than 5 percent probability that the difference could have 
occurred by chance with samples of this size. 

Second, there is a much greater potential for sample bias at Program B than at Program A 
because fewer patients consented to followup and fewer of those for whom followup attempts 
were made were successfully contacted. Actually, the number of patients from both programs 
who reported abstinence is identical—40. It cannot be determined from the data available which 
program is superior. It depends on how the nonlocated patients are doing. 


System questions are generally those for which answers can be enacted into statewide policy, 
such as standardized patient placement, continued-stay criteria, and resource allocation. 
Examples of such questions are: 

• Can the higher costs associated with inpatient or residential treatment be justified in terms of 
superior outcomes? For which kinds of patients? Under which conditions? 

• What is the optimal intensity of outpatient treatment for different patients at different stages in 
the recovery process? How many days a week? How many hours a day? 

• Can an optimal length of stay be determined based on patient characteristics or treatment 
progress? Is there a minimum length of stay, short of which treatment is of no value? Is there a 
maximum beyond which no further benefits accrue? Should treatment last much longer than is 
currently the case but perhaps have greater flexibility for crisis intervention and decreasing 
levels of involvement? 







• What ancillary or adjunct services are essential to improved treatment outcomes for which 
groups of patients? Housing assistance? Education or job training? Childcare? Parenting skills 
training? Transportation? 

• Are specialized programs for special populations associated with better outcomes for certain 
groups of patients (women, adolescents, racial/ethnic or other cultural groups, cocaine abusers, 
alcohol abusers, persons with psychiatric disorders or histories of crime, for example. If so, are 
the outcomes so superior as to justify a greater investment of resources in such programs? 

• Are certain therapeutic modalities associated with better outcomes for some kinds of patients 
rather than others? 

Two examples of the way statewide aggregated data can be used to address broader questions 
illustrate the benefits of this approach to system design. (See the Hypothetical Example boxes on 
the pages that follow.) Each example, while hypothetical, illustrates a practical and feasible 
accomplishment of a State OMS. However, this is not to suggest that getting to this point of 
OMS capability is a simple task. 

The next section includes some fundamental issues that need to be addressed early in planning an 
OMS. 


Guiding Principles 

Once a commitment is made to developing a State outcomes monitoring system—once the idea 
is sold—there is often a temptation to move immediately to designing data collection forms. But 
the process demands a great deal of thoughtful reflection before any concrete design issues can 
be addressed. Some basic realities that must be considered are briefly discussed here as guiding 
principles. 

The nature of addiction must be considered in defining beneficial outcomes. Persons with 
AOD problems are diverse in their personal characteristics and life histories, their substance use 
patterns, the adverse consequences they have experienced, and the resources they bring to bear 
on their attempts to recover from addiction. In recognition of this diversity, it is essential to 
measure treatment outcomes in terms of each individual patient's improvement along a 
continuum rather than in comparison with some predetermined arbitrary success measure. A 
dichotomous measure of success versus failure is too simplistic to characterize treatment 
outcomes (Longabaugh, 1991; Stinchfield et al., 1994b; Wells et al., 1988). 

Many patients will improve sufficiently to justify the investment in treatment even when they do 
not attain ideal treatment goals (McLellan et ah, 1993; Walsh et ah, 1991). Because AOD abuse 
leads to both health and social problems, treatment outcomes measurement must include many 
dimensions. Measures must address AOD use as well as the areas in which AOD use has had an 
impact on patients' lives, such as general health, social and occupational functioning, and legal 
involvements. 


The specific outcomes measures selected should be derived from the definition of treatment 
benefits and related to the goals of treatment. At the same time, it is imperative to foster a 
climate of realistic expectations regarding what treatment can and should be expected to do. For 
example, to the extent that AOD use impairs job performance and social functioning and leads to 
crime, beneficial treatment can be expected to show concrete effects in these areas. But treatment 
in its limited timeframe cannot be expected to compensate for economic deprivation, educational 
deficits, and other longstanding difficulties. 

The OMS must be feasible. Designers of a successful outcomes monitoring system will have to 
balance practicality against the ideal design. Availability of resources will dictate the scope and 
capabilities of any system. AOD resources are shrinking in many areas, even while demands 
placed upon them are increasing. As time goes on, more is expected of State agencies and 
treatment providers. In this climate, outcomes monitoring may be seen as just one more added 
burden. Without a distinct and visible payoff, enthusiasm for the undertaking is likely to be 
lacking. 

An outcomes monitoring system must be acceptable to a broad spectrum of stakeholders— 
especially treatment program administrators and staff who will be asked to collect much of the 
required data (Camp et ah, 1992). The final design must minimize treatment program disruption 
and demands on staff. At the same time, its findings must be credible and useful to its audience. 
The design must ensure valid data and generalizable results. 

Costs of an ideal outcomes monitoring system would make it prohibitive. Therefore, planners 
will have to agree to accept the limitations of their OMS. No set of instruments can measure 
everything everyone wants measured. Nor will any set of procedures live up to the demands of 
all researchers. More data collection and more stringent methods translate into more time and 
money expended (Longabaugh, 1991). 

Tough decisions will have to be made at the outset regarding the essential components of the 
system. Which information is most useful? What methods are most practical? What followup 
intervals and procedures are feasible? What can be done well within a limited budget? What can 
be done first? Would it make sense to implement a skeletal system and supplement and improve 
it later? 

Not all treatment program goals are suitable for use as outcomes-based accountability measures. 
While certain personal changes have profound meaning for individual patients, they may be 
difficult and time consuming to measure. The most useful outcomes measures will have broad 
public and political appeal, such as evidence of reduced harm to society and cost savings in other 
areas that result from successful treatment (Alio et al., 1988). 


Hypothetical Example: Poorer Outcomes Found for Rural Patients 

Suppose an analysis of aggregated statewide program data revealed that patients 







admitted to treatment programs in rural areas had poorer outcomes than those 
admitted to programs in urban areas, even when differences in patient characteristics 
and treatment services were taken into account. Further analysis revealed that the 
differences in outcomes were largely attributable to the use of community support 
services after treatment: on average, urban patients reported much greater 
involvement with community resources than rural patients. In discussions to interpret 
this finding, treatment staff consistently emphasized the relative lack of recovery 
support services in remote areas. This lack led to a recommendation to allocate more 
State resources toward increasing the availability of community supports in rural 
areas. 

Without this analysis of aggregate statewide data, an erroneous conclusion might have 
been reached and resources wasted. If the focus had been on individual program 
comparisons, the evaluation might have concluded that rural treatment programs were 
of poorer quality than urban AOD abuse treatment programs. A great deal of time and 
money may have been spent to improve rural programs when this improvement was 
not what was needed. 


A successful OMS requires some degree of standardization across participating treatment 
programs. In order to compare patients, treatment services, or outcomes, some degree of 
standardization will be required. Some consensus must be achieved with respect to baseline and 
outcomes measures and the use of data collection instruments and procedures. While planners 
may agree in principle, the difficulty arises when specific decisions must be made. Currently, 
there are no instruments that have universal acceptance for purposes of patient assessment or 
treatment service description (Longabaugh, 1991). Without a directive for certain standardized 
data elements or procedures superimposed by the Federal Government, States will be on their 
own to reach internal consensus. While there may be benefits to having all States adopt at least a 
subset of identical data elements to measure outcomes, no mechanism exists to ensure this 
standardization. 

General recommendations regarding OMSs must take into account the diversity of States. 

A great deal of diversity exists in the States and territories that review the recommendations of 
the consensus panel presented in this TIP. These units vary in terms of the populations they 
serve; the prevalence of alcohol and other specific drug use disorders; the rates of injection drug 
use, FIIV/AIDS, and tuberculosis among the AOD-using populations; the mix of rural and urban 
communities; poverty rates; language diversity; the racial or ethnic composition of the populace; 
and the extent to which cultural factors influence substance use patterns and attitudes toward 
treatment. 

Differences also exist among States in terms of their financial resources and the burdens placed 
upon these resources. States also vary in terms of how programs currently operate and how they 







are funded, as well as in the variety of treatment philosophies, programs, and services they have 
in operation. 

There are vast differences among States in terms of their current MIS capabilities and how much 
must be accomplished before an outcomes monitoring system can be implemented. While some 
States have already designed and put into practice statewide data collection systems that include 
a followup component, others have yet to begin the planning process. 

Finally, it is important to acknowledge the ever-changing landscape of the healthcare delivery 
system. Any general recommendations with regard to AOD abuse treatment outcomes 
monitoring systems must be considered in the context of other State or Federal requirements that 
may be imposed. 

An OMS is not successful unless its findings are put to practical use. The most frequent and 
justifiable complaint about large-scale data collection requirements is that the information 
disappears into a black hole. What too often happens is that the bulk of planning effort and 
financial resources are committed to data collection. Data analysis and the dissemination of 
results receive short shrift. Unless everyone whose efforts are needed to ensure success is 
convinced that something useful will emerge from all their hard work, the process will fizzle out. 
Initial planners must consider in concrete ways how the resulting information will be used and 
what benefits it will have. They must determine who will use the results. 


Hypothetical Example: Recovery Maintenance Services Found To Be a 
Strong Predictor of Successful Outcomes 

Suppose an analysis of aggregated statewide program data found that one of the strongest 
correlates of successful outcomes is length of involvement in program-directed continuing care 
or recovery maintenance services. In this hypothetical analysis, continued contact with 
professional services increased the likelihood of abstinence even when patients were attending 
AA or similar groups. In fact, participation in continuing care after primary treatment was 
completed was found to be a stronger predictor of successful outcomes than any specific 
component of primary treatment itself. A recommendation emerged to prioritize the use of 
limited available resources to promote the development and expansion of continuing care rather 
than to improve or add intreatment service components. 


Treatment program administrators and staff? Referral agents? Legislators? Consumers? Will 
findings be used to reward high performers and penalize poor perfonners? Will findings be used 
to shut down programs? Will the results be the basis of performance-based contracting? Will 
results be used as the basis for recommendations to match patients to programs or services? 
Achieving the buy-in and turning empirically derived information into policy are addressed in 
detail in Chapters 2 and 8. 

Progress can be incremental. The perfect outcomes monitoring system cannot be designed and 
implemented in the abstract. The process may have to begin on a small scale with a limited 







number of data elements and/or a limited number of participating programs. Unanticipated 
difficulties will occur. Revisions will be required. Some questions will not be answered. 
Improvements can be made as time goes on. The scope can be increased. States can learn from 
their own successes and shortcomings—and they can learn from the efforts of other States. 


Organization of This Volume 

Chapter 2 examines the political considerations necessary to plan a successful outcomes 
monitoring system. This chapter addresses the involvement of a variety of stakeholders, the 
formation of steering committees, and the obstacles and anxieties that will be encountered. 
Chapter 3 addresses ethical issues related to outcomes monitoring systems. 

Chapter 4 examines a variety of methods appropriate to statewide outcomes monitoring systems 
and discusses the benefits and limitations of various strategies. This chapter includes some 
specific recommendations as to patient samples, program samples, patient followup data 
collection methods, followup intervals and procedures, and program staff training. Chapter 5 
focuses on the content of an OMS: the information to be collected to describe patients at baseline 
and followup and to describe treatment. Chapter 6 addresses legal issues relevant to outcomes 
monitoring systems. 

Chapter 7 addresses some technical considerations with respect to management information 
systems and appropriate hardware, software, and data transfer along with staffing and budgetary 
concerns. Chapter 8 is devoted to the dissemination of findings and turning empirical results into 
policy. 

Appendix A is the bibliography, containing all references cited in the text. Appendix B describes 
current State outcomes monitoring systems, and Appendix C provides a list of representatives of 
Federal agencies and national professional associations who participated in the Federal resource 
panel. The latter group provided advice in the early stages of planning the CSAT consensus 
panel and the TIP. Appendix D provides the names and affiliations of all those who reviewed 
and provided comments on the first draft of the TIP; their advice contributed greatly to the final 
shaping of the document. 


Chapter 2—Political Considerations in 
Planning a Successful OMS 

As noted in the introduction to this Treatment Improvement Protocol (TIP), current efforts by 
single State agencies (SSAs) to address the issue of outcomes accountability are occurring in the 
context of healthcare reforms that may produce profound changes in the delivery and 
reimbursement of substance abuse treatment services. While uncertainty about changes in the 
healthcare system has the potential to complicate the process of planning and implementing a 
State outcomes monitoring system (OMS), it need not—and should not—deter SSAs from 
developing of such a system. Indeed, various State reforms are already placing new requirements 
on SSAs to conduct outcomes monitoring. 

Given the inevitability of such requirements, there is a clear advantage to the SSA in taking the 
initiative to develop an OMS that will meet its own identified information needs as well as 
satisfy externally imposed demands for accountability. In doing this the SSA will not only obtain 
data essential to improving treatment outcomes, but will also strengthen its position as the lead 
agency for monitoring substance abuse treatment services within the State. In leading the 
development of an OMS, the SSA will help ensure that the data collected and the uses to which 
they are put meet the needs both of society as a whole and of the vulnerable, high-risk groups 
that are primary users of publicly supported alcohol and other drug (AOD) treatment services. 
The purpose of this chapter is to assist the SSA director and staff in understanding the process 
for gaining support for a State OMS that will meet these dual needs. 


There is a clear advantage to the SSA in taking the initiative to develop an OMS that meets internal 
and external needs for information. 


Social Value of AOD Treatment and an Outcomes Monitoring System 

Outcomes monitoring systems must be relevant and acceptable to a broad spectrum of 
stakeholders, including the following: 

• Consumers of AOD treatment services and their family members 

• Patient advocacy groups 

• Program administrators and clinical staff 

• Organizations of counselors and other professional providers 

• Other State service systems, including human services such as public health and criminal justice 

• Employers 

• Third-party payers 

• Taxpayers 









• Legislators 

• State governors and other representatives of the executive branch of State government. 

Support from all of these groups is needed to ensure 1) development of the OMS, 2) appropriate 
and effective use of the resulting data, and 3) achievement of desired policy changes. The 
interests that these stakeholders represent and how those interests can be addressed in building 
support for the OMS are discussed later in this chapter. 

The principles and values of these diverse stakeholders influence the nature of AOD treatment 
services provided through the SSAs and have important implications for how an OMS is 
developed and the kinds of data that will be gathered. To develop an OMS that responds to data 
needs within the State and to secure support for its development, it is essential to identify and 
assess these principles and values in the early planning stages. 

For example, one important rationale for undertaking outcomes monitoring is to achieve good 
treatment outcomes. A commitment to protect vulnerable and at-risk groups as part of 
government's social contract with the people is another. Ensuring the most effective use of 
limited resources is yet another value driving the development of an OMS in many States. 

Historically, value judgments about addiction have often led to derailment of efforts to address 
alcohol and other drug abuse problems in the political arena. Clarity at the State level about the 
belief system and values of key policymakers such as the governor or legislative committee 
chairs will allow the SSA to promote outcomes monitoring effectively. 

The link between substance abuse and a host of social and public health problems—family 
disintegration, crime, injuries, lost productivity, premature morbidity and mortality—demands 
that government have a role in the prevention and treatment of AOD addiction. This argument 
constitutes a powerful basis for garnering broad-based support for AOD treatment programs 
from public policymakers and their constituents. 


Historically, value judgments about addiction have often led to derailment of efforts to address 
alcohol and other drug abuse problems in the political arena. Clarity at the State level about the belief 
system and values of key policymakers such as the governor or legislative committee chairs will allow 
the SSA to promote outcomes monitoring effectively. 


Given the public policy importance of this link between substance abuse and other social and 
health problems, perhaps the most compelling measures of AOD treatment success will be 
demonstrated by decreases in these problems. While treatment programs cannot be held 
accountable for achieving such improvements singlehandedly, they clearly have an important 
role to play in conjunction with other health and social service agencies. SSAs have a 
responsibility to forge partnerships with these other agencies and develop with them a collective 
vision for how common programmatic goals and objectives can be met and measured. 







Outcomes to be measured should include those that will justify, in terms of societal outcomes, 
the commitment of resources to treatment. These outcomes include potential cost savings from 
AOD treatment that will result from decreased demand for other services such as incarceration or 
foster care. Data exist that can demonstrate these cost advantages of treatment to society, and this 
type of cost-effectiveness data must be emphasized in a State OMS. At the same time, an agency 
must set realistic expectations and emphasize the interdependence of treatment success with 
improvements in other aspects of social welfare. 


Other Factors for Consideration in Initiating an Outcomes Monitoring System 

A number of factors may influence not only the form that outcomes monitoring should take, but 
also the process by which the OMS will be developed. While each State will have its own unique 
factors that must be taken into account, some factors are present in virtually all States. However, 
variations in the States preclude their using universal strategies for dealing with these common 
factors. The intent of this section is to ensure that these factors are considered and their potential 
for influencing the success of outcomes monitoring is assessed. 

Healthcare reform. Healthcare reform may produce a number of results that will affect the 
delivery of AOD treatment services. In a climate of limited resources and aggressive marketplace 
competition, the SSAs may be asked to justify the retention of specific services for some client 
populations. By demonstrating the importance of these services to the general public welfare, 
OMS data may help these programs establish themselves as valuable "niche" providers in the 
reformed healthcare delivery systems of their States. 

Trend toward merging AOD and mental health services. Expectations of treatment outcomes 
for the severely mentally ill are quite different from those for the recovering substance abuser. In 
cases where services are integrated or merged, care must be taken to ensure that distinctions 
among patients are retained in data collected for an OMS. Accurate information about patient 
status at intake, for example, is critical to establishing realistic expectations for outcomes and 
appropriate measures of success. 

Budgetary resources. Within a State's AOD treatment system, development of an OMS will 
probably compete with existing State and Federal priorities for allocation of limited funds. It will 
be necessary to convince policymakers of the value of shifting funds from existing budget 
priorities to outcomes monitoring, a shift that may be made more difficult by the tendency of 
many at the policymaking level to view issues only in terms of short-term political expediency or 
payoff. 

It will also be essential for the SSAs to convince programs that the diversion of treatment funds 
to support outcomes monitoring, if necessary, will ultimately pay off by demonstrating the value 
of AOD treatment and improving outcomes. Building in performance and funding incentives to 
programs that encourage outcomes monitoring may be one way of accomplishing this. The 
possibility of collaborative funding efforts with other provider agencies may be another. 


Technological resources. Implementation of an OMS will require substantial technological 
resources, and limitations on those resources are among the pragmatic considerations that will 
affect the design of an OMS. While ideally all providers would use compatible computer 
hardware and software, in reality, multiple hardware and software systems already exist within 
most provider networks. Moreover, the startup costs associated with a fully integrated system 
would be prohibitive for most agencies. 

From a practical standpoint, existing systems cannot be abandoned but must be factored into 
operational plans for the OMS. Thus, while SSAs are encouraged to develop long-range plans for 
such an integrated OMS—to be implemented in phases based on availability of resources to 
providers—they must be aware that insisting on specific hardware and software requirements for 
providers at the outset will only undermine the likelihood of implementing the system. 

Training and technical assistance. Substantial amounts of training and technical assistance may 
be necessary for providers to undertake desired data collection. Both staff and budgetary 
resources for this effort must be identified. 

State political environment. The importance of identifying key players within the State's 
political environment and developing a strategy for obtaining their support has already been 
suggested. The identity of these players may vary considerably from State to State and may be a 
function of personalities as well as governmental organization. The SSA's relationship to these 
key political players will also vary. In some instances, the SSA director serves at the pleasure of 
the governor. In all cases, the SSA’s cultivation of political allies and effective perfonnance as an 
advocate on behalf of AOD treatment services will be critical to its success in gaining support for 
its outcomes monitoring proposal. 

Organization of service delivery systems within the State. Structures within State 
government, their relationship to treatment service providers, and issues of local and State 
control will clearly influence the developmental process of an OMS. Limited statutory authority 
of the SSA in some States, for example, will require that the agency find other ways of gathering 
needed support for OMS implementation. The agency that does not have sufficient authority or 
leverage to muster support must develop relationships, networks, and incentives to foster 
progress toward developing the OMS and at the same time develop and implement strategies to 
obtain greater authority within the State's decisionmaking process. 

Assumptions underlying current program policies. State treatment programs inevitably reflect 
a set of assumptions that may or may not be validated by the results of outcomes monitoring. The 
implications of such findings should be considered. A basic assumption is that treatment works, 
and treatment staff may be reluctant to develop a monitoring system that could call this 
assumption into question. Another common assumption is that a range of services with differing 
levels of intensity is appropriate to meeting the treatment needs of diverse clients. Many 
providers also assume that the only way to have good outcomes is through treatment. 

Failure to recognize the assumptions operating within an agency or program can undermine 
development of an OMS and its usefulness as a planning and accountability tool. Consideration 
should be given to the appropriate interplay between such assumptions and plans for outcomes 



monitoring. A primary strategy for identifying assumptions operating within the State's treatment 
services will be a careful review of plans, policies, and statutes of both the SSA and the various 
treatment providers. The perspective of advisory groups may also be useful. 

Rationing of healthcare. Rationing of care as a consequence of future cost containment 
measures is a frequently expressed concern in the context of healthcare reform. In fact, 
inconsistencies already exist between care that is reimbursable and care that is needed to achieve 
reasonable therapeutic goals. In some areas, resources are allocated to serve more patients with 
short stays, rather than fewer patients with longer stays. This allocation is a clear example of the 
rationing of care. How this rationing occurs and its prevalence should be considered in 
developing an OMS, since limitations on treatment services obviously influence outcomes. 

The OMS can be designed to determine the relationship of setting and length of stay to 
outcomes, but if treatment is severely restricted throughout the system, the observed outcomes 
may be uniformly poor. The greater the variability in factors such as setting and length of stay, 
the more likely it is that recommendations can result with respect to optimal service levels. 

Threats to the status quo. Outcomes monitoring may expose program weaknesses, and 
treatment providers may be understandably concerned about the impact of such findings. 
Emphasizing to providers that the focus is on improving outcomes—and not on penalizing 
programs—should help alleviate their concerns about negative repercussions of outcomes 
monitoring. Further suggestions on how to respond to the concerns of providers are contained 
later in this chapter in the section on building support for the OMS. 

Political ramifications. Negative findings of the OMS will inevitably generate political 
ramifications. Some treatment programs or program types may enjoy strong political support, 
and results of outcomes monitoring may threaten their survival. Efforts may be mounted to 
defend treatment services that do not fare well in outcomes monitoring but have strong political 
or private support. Individuals who themselves have benefitted from particular treatment 
modalities may be defensive or threatened by findings that challenge the presumed value of 
services that they favor. 

Needs and concerns of agency staff. Credentialing is increasingly important within the AOD 
treatment field, and the results of outcomes monitoring may point to the need to upgrade the 
qualifications and credentials of treatment staff. If so, any training required to accomplish this 
will entail additional expense. Linking funding to the presence of qualified, credentialed staff 
may motivate programs to undertake the training and preparation necessary to meet credentialing 
requirements. 

Summary 

This brief review of factors influencing the development of an OMS is by no means exhaustive; 
however, it does serve to make the point that outcomes monitoring cannot be done in isolation. 
Rather, monitoring must occur within the context of State politics and programs and may also be 
influenced by factors well beyond the purview or control of the SSA or the State. Such factors 
should not be viewed as barriers that preclude the development of a useful OMS, but as 


influences that will shape the process by which it is developed and its purposes and design. 
There should be general agreement on the ultimate goal of outcomes monitoring—improved 
outcomes for people who enter substance abuse treatment, their families, and society—and its 
importance should serve as a powerful motivator to the cooperation and compromise that are 
necessary to achieve it. 


Factors to Consider In Initiating an Outcomes Monitoring System 

• Values of the stakeholders 

• Changing healthcare delivery systems 

• Trend toward merging AOD abuse treatment and mental health services 

• Budget priorities 

• Limitations on technological resources 

• Need for training and technical assistance 

• State political environment 

• Structures within State governments 

• Assumptions operating within an agency or program 

• Rationing of healthcare 

• Threats to the status quo 

• Political ramifications of negative findings 

• Concerns of agency staff. 


Building Support for the Outcomes Monitoring System 

Successful development and implementation of a State OMS will depend in part upon the degree 
to which key constituencies and interest groups within the State are involved in the planning and 
development process. The participation of a wide range of interested parties can both enhance 
the design of the system and provide a broad base of support for its operation and use. The 
following mechanisms and strategies can help obtain this participation and support within the 
context of a systematic planning process. 

Who Should Be Involved? 

The issue of substance abuse is a community concern, and a wide range of participants can 
contribute to the success of an OMS as well as benefit from the information it produces. In 
general, the OMS planning process should include people with the following characteristics: 

• May be affected as a result of information generated by the system 

• Will be involved in the data collection 

• May be able to improve the quality of the system or the data collected 








Can develop a network of support for implementation of the system once it is developed. 


Specific groups to be considered include: 

• Policymakers with responsibility for planning and implementation of AOD abuse treatment 
and other health and human services. These policymakers may be State agency heads or 
administrators to whom such responsibilities have been delegated. They have a shared interest 
in social issues, can offer the perspective of other services in the identification of appropriate 
data to be collected, and will be key collaborators in any efforts to integrate benefits and 
services. 

• Treatment program administrators and clinical staff. As providers of substance abuse 
treatment services, this group will be responsible for most of the data collection; their support is 
critical to the success of those efforts. Moreover, they will bring a sense of realism to the 
consideration of what data can and should be collected, how they can be collected, and how 
data can be used by programs to improve outcomes. Finally, by their career choice of working in 
the treatment field, they have demonstrated a personal investment in successful treatment 
outcomes. 

One key to obtaining provider support for outcomes monitoring will be identifying 
outcome measures that programs are willing to accept and use as a basis for program 
decisions. These outcomes must be considered fair measures of program effectiveness 
and be developed in consultation with providers and clinicians, and the data collected 
must be considered adequate to provide a basis for judgments or decisions. 

This concept of fair measures has important implications for the design of the OMS, the 
purpose it serves, and the outcome variables that will be the focus of data collection 
efforts. Expectations about the kind of information that outcomes monitoring can provide 
and the uses it can serve must be realistic. In most States, outcomes monitoring systems 
or data probably will not be adequate for States to make definitive statements about 
specific programs. However, data and the OMS should enable the SSA to assess what 
strategies are working best across the State. Enough data on services must be provided to 
allow a determination of what mix of services seems to work best for whom. Programs 
can then be directed toward provision of the best service mix for different kinds of 
patients. 

• Other health professionals. The cooperation of health professionals from a variety of disciplines 
and specialty areas will be important to establishing linkages among services, and their cross- 
disciplinary perspectives will help ensure that the scope of the proposed data set is not defined 
too narrowly. 

• Criminal justice system personnel. Members of the criminal justice system—including judges, 
corrections personnel, law enforcement representatives, and providers of legal services—have 
an important perspective on substance abuse and its societal impact. 

• Regional, county, and other local health and social service agencies. The goals of one agency 
cannot be met without the cooperation of collateral service agencies. Other health and social 
service agencies, such as departments of housing and departments of public health provide 
collateral services necessary for clients in AOD treatment. 



• Benefits managers and managed care representatives. As purchasers of AOD services, 
members of this group have a vested interest in obtaining outcomes data in order to set 
appropriate service criteria. They may already have outcomes data of their own to offer and 
experience in collecting them. They also provide an important link to the private payer 
community. 

• State legislators and their staffs. Of particular importance are legislators serving on committees 
with oversight responsibility for AOD services or other functions related to operation of those 
services, such as budget committees. They have a major impact on policy decisions and are in a 
position to alter and direct financing. 

• Special populations. A broad-based consumer perspective can be obtained through inclusion of 
representatives of populations traditionally underserved, such as people of color, recent 
immigrants, prisoners, the developmentally disabled, mentally ill persons, pregnant women, and 
youth. 

• Researchers and other experts in OMS design and implementation. The assistance of technical 
experts clearly can enhance the development and implementation of an OMS. Inclusion of these 
experts in the early stages of planning ensures their understanding of the purposes of the OMS 
and the myriad considerations that influence its final design. 

The support and long-tenn commitment of these groups will depend upon the extent to which 
they become convinced of the relevance of the OMS to their own interests and goals. Gaining 
this support will require a process of cultivation much like that associated with community 
development and grass-roots organizing. Later sections of this chapter discuss how this support 
can be accomplished through formal mechanisms such as a steering committee or workgroups. In 
addition, a review of the literature on community organizing may be useful to gain an 
understanding of the procedures, strategies, and skills involved in this process. 

Organizing the Planning Process 

A systematic, collaborative approach to planning and developing the OMS will maximize 
prospects for its successful implementation. The approach outlined here emphasizes the formal 
involvement of organizations and groups already identified as important collaborators. Within 
the planning structure, the SSA serves a coordinating function and provides overall leadership 
for the planning process in the person of the project director. Additional staff resources will also 
be required. A steering committee and a series of task-specific workgroups provide mechanisms 
for gathering the assistance and support of other agencies and organizations. 

Appointment of a Project Director 

As the person charged with overall responsibility for development and implementation of the 
OMS, the project director must also have the authority necessary to meet that responsibility. In 
addition to understanding the intended uses of the OMS and the principles of program planning 
and evaluation, the project director must possess strong interpersonal skills and sound political 
instincts. He or she must be able to cultivate and maintain good working relationships with the 
leadership of all groups targeted for participation in the planning process. Establishing good 
communication channels with these groups should be one of the earliest steps in the planning 
process, and the success of this coalition-building will be key to OMS development, as will the 
ability to analyze and negotiate the political process. 



These are other important attributes of a project director: 

• Knowledge of basic research skills 

• Knowledge and understanding of the position of the SSA within the State and its relationship to 
providers and other social service agencies 

• Skills in group facilitation and consensus development 

• Ability to gain respect and credibility with policymakers and other administrators of the agencies 
involved. 

Formation of a Steering Committee 

A steering committee that represents participating groups is recommended as a fonnal 
mechanism for obtaining the input of those groups. The members of this committee should be 
key decisionmakers within the organization or group they represent. Following are the primary 
responsibilities of the steering committee: 

• Develop a mission statement and vision of the OMS 

• Establish goals for the OMS 

• Provide oversight for OMS development and operation. 

Use of Working Groups 

In addition to forming a steering committee, it may also prove useful to establish workgroups 
that will supplement efforts of the steering committee and assume much of the nuts-and-bolts 
responsibility for developing the OMS. Workgroup members, usually staff members within their 
respective organizations, should be selected by the steering committee. Specific workgroup 
assignments that might contribute useful infonnation include: 

• Framing objectives for the OMS, based on goals established by the steering committee 

• Conducting a needs assessment 

• Identifying research questions to be addressed by the OMS 

• Developing a work plan and implementation strategies 

• Reviewing current data sources 

• Identifying a core set of data 

• Outlining a research methodology. 

Identifying Roles, Responsibilities, And Tasks 

Within the collaborative framework created to guide the planning process, it will be important to 
delineate roles and responsibilities of each participant and to specify tasks and deadlines for their 
completion. Assignment of responsibilities will reinforce the participation and commitment of 
members, while contributing to the work of developing the OMS. Careful attention to planning 
meetings—for example, setting an agenda and identifying effective facilitators of the group 
process—will be important to gain maximum benefit from the group. A process for conducting 
meetings should be developed that will pennit expression of divergent opinions without allowing 
these differing viewpoints to dominate proceedings. Work plans that identify who is doing what 




and projected dates of completion are a practical tool for ensuring common understanding of 
roles and responsibilities. 

Establishing Goals and Ground Rules 

Agreeing on common goals for the OMS is essential to obtaining commitment to the effort from 
all participants in the planning process. The buy-in that these shared goals can produce will be a 
key element in achieving success. When differences of opinion occur, the presence of these 
common goals for the OMS will help keep planning on track and maintain unity of support. 

Equally important to establishing common goals is reaching agreement on the ground rules for 
group decisionmaking and detennining how consensus will be defined. Whenever priorities are 
set or choices made, there is the likelihood that the interests or needs of some groups will not be 
served. If there is agreement on the purpose and goals of the OMS and if the agreed-upon 
process for development is adhered to, then participants should agree to support the resulting 
system plan, even if it does not satisfy all of their own requirements. 

Experience in Minnesota offers a useful example of how this support can be achieved. The 
collaborative process used there to develop an OMS included an agreement by participants that 
they would accept group consensus regarding data priorities, even if their own preferences were 
not adopted. Consensus in this circumstance did not mean total agreement, but acceptance and 
support for the larger group decisions. 

Maintaining Ongoing Involvement 

A variety of strategies can be employed to maintain participation and commitment among 
participants in the developmental process: 

• Meeting periodically to review progress and address problems 

• Providing tangible evidence of progress and results on a regular basis 

• Maintaining contact with key decisionmakers in each group, even though they may have 
delegated tasks to others. 

Identifying Specific Questions To Be Asked by the OMS 

Specifying questions to be asked by the SSA's outcomes monitoring system is a dynamic 
process. The questions will vary from State to State and over time. Even though an identified 
data need may provide the initial impetus for development of the OMS, the questions the OMS is 
designed to answer will undergo refinement during the process of development and consensus 
building. Establishing parameters for the types of questions that may be considered will be 
facilitated by the earlier development of a mission statement and agreement on goals of the 
OMS. 

An information needs assessment may be undertaken to identify the key questions by each of the 
collaborating groups. The steering committee may then analyze the results of this needs 
assessment to establish priority questions that should be addressed. In establishing priorities 



among questions, it may be necessary to weigh the relative needs of the various groups. Prior 
agreement on the definition of consensus will become critical to winning acceptance of these 
priorities. 

A more detailed discussion of identifying questions for the OMS can be found in Chapter 5 of 
this TIP. 

Assessing Current Capacity To Answer Priority Questions 

Once priority questions have been agreed upon, the State's current capacity to address the 
questions must be assessed. This process should not be limited to an examination of the 
information resources of the SSA. An inventory of other agencies' data collection efforts will be 
needed, a project requiring interagency cooperation and information sharing. Such an inventory 
must consider not only the data elements collected, but also the accuracy of the data and the 
utility of its format for SSA purposes. 

In addition to examining existing data collection efforts, an assessment of the SSA’s staff 
capabilities and available technology is also required. Some State agencies may not have the 
capability to undertake outcomes monitoring and will want to contract this function to another 
agency or institution. 

Agencies will have limited resources for conducting outcomes monitoring, and the cost of 
collecting each data element must also be considered. Where data collection costs will be high, 
the importance of the infonnation must be weighed against the costs to justify inclusion in the 
OMS. The results of these calculations may necessitate a reordering of priorities if this informal 
cost/benefit analysis does not meet guidelines or expectations regarding usefulness and 
importance of data to be collected. 

Requirements must be determined for training SSA and program staff to enable them to use the 
data and for training the provider network staff to enable them to collect the data. 

Assessing Potential Usefulness of External Databases and Instruments 

Every effort should be made to avoid duplicating existing data collection activities or reinventing 
existing systems or instruments that have already been validated and may meet local needs for a 
data collection instrument. In addition to freeing resources for other uses, a valid instrument 
common to other groups allows for comparability of findings. If such instruments can be adapted 
to SSA data collection needs, the resulting outcomes data on substance abuse would then parallel 
data collected in other areas, an added benefit. 

Identifying Data To Be Collected by The OMS 

Based on results of the State capacity assessment, it should now be possible to specify those 
priority data elements for which no other source exists and which should be collected by the 
OMS. It may be desirable to identify a core set of data that will be able to answer whatever the 
priority questions may be at any particular time. From these data will emerge information on 



services, patient characteristics, social indicators, and other variables that can demonstrate the 
value of substance abuse treatment to legislators and other policymakers. In addition, if decisions 
about priority information needs have been reached through a collaborative process with other 
data users, then data generated by the OMS should be relevant to the network of potential users 
who developed it and whose continued support for the OMS is essential. 


Tasks in Successful Development of a State OMS 

• Organize the planning process 

• Identify roles, responsibilities, and tasks 

• Establish goals and ground rules 

• Maintain commitment among participants 

• Identify specific questions to be asked by the OMS 

• Assess current capacity to answer priority questions 

• Avoid duplicating existing data collection activities 

• Identify data to be collected by the OMS. 








Chapter 3—Ethical Issues and 
Outcomes-Based Accountability 

The purpose of this chapter is to ensure that a discussion of ethics and the values choices raised 
in alcohol and other drug (AOD) treatment policy are part of the framework for monitoring the 
outcomes of treatment. This chapter will not resolve the ethical dilemmas that may occur when 
monitoring outcomes, but it will attempt to identify and describe some of these dilemmas. 

Policymakers and practitioners in the field must be able to discuss ethical choices openly rather 
than burying them beneath more technical activities or ignoring them altogether. Values matter, 
and clarifying what values are under discussion can help make policy decisions easier to 
understand. Policy can be defined as the authoritative allocation of values, and it is important to 
be clear which values are under review. 

Ethics as a field of study is useful in assessing both the rightness of decisions and the fairness of 
the decisionmaking process. Ethical guidelines can help order values when values are in conflict. 
The use of ethical concepts does not dictate final decisions, but can help substantially in 
clarifying what criteria are being used and how they are weighted in comparison with each other. 
A classic ethical issue, for example, occurs in the field of substance abuse treatment when there 
is a need to weigh the individual's freedom to choose against the rights of others (such as the 
individual's children or the larger society) not to be harmed by choices made by the individual in 
the role of parent or citizen. 

Despite the frequent use of the phrase "the ends don’t justify the means," ethical choices often 
involve exactly this kind of weighing of different ends and judging whether or not coercive 
means are justified by the probable results. Classical and recent ethical analysis has focused upon 
the neutrality needed for an observer to make such judgments. John Rawls's landmark work A 
Theory of Justice shows the importance of distributing goods to benefit the least advantaged in 
society, based upon a neutral observer's decision about which individuals fall into that group 
(Callahan and Jennings, 1983; Rawls, 1971; Rhodes, 1986). 

Other ethical issues in human services include the point of view of the patient compared with 
that of the treatment staff and the funder; the competence of patients to make decisions about 
their own well-being; and the power of those who exercise coercive controls, and how doing so 
affects their perspective (Callahan and Jennings, 1983; Rawls, 1971; Rhodes, 1986). 

Ethical choices involve the following: 

• The rights and responsibilities of the patient, including the expectations society places upon 
those individuals whose behavior it is attempting to alter 

• The rights and responsibilities of the treatment system and those who work in it 

• The validity of claims made by funders and other stakeholders on both the treatment system 
and on patients, judged by some clear, specified standard of what is fair. 



The Ethics of Outcomes-Based Accountability 


The last few years have brought a renewed interest in using outcomes measures to increase the 
accountability of human services programs. New Federal and State legislation has elevated the 
visibility of outcomes by linking funding regulations to outcomes achieved. But these are not 
merely issues of good government, program design, evaluation science, or technical 
measurement. These issues of outcomes measurement have deep ethical content. 


The last few years have brought a renewed interest in using outcomes measures to increase the 
accountability of human services programs. New Federal and State legislation has elevated the 
visibility of outcomes by linking funding regulations to outcomes achieved. But these are not merely 
issues of good government, program design, evaluation science, or technical measurement. These 
issues of outcomes measurement have deep ethical content. 


The first set of outcomes-related issues concern who is acting and who is being acted upon in 
service delivery. There is a distinction between the person as subject and the person as object. 
The person as subject acts; the person as object is acted upon. One confers dignity when treating 
another as a subject, but to treat another as an object is demeaning. This distinction between 
subject and object argues powerfully for what some have called "empowerment" and expanded 
patient involvement. But as important as these concepts are, they result in changes that may be 
restricted to the processes of programs rather than to their content. Central to outcomes 
monitoring is the idea of patients as subjects. Such a focus argues that the content of AOD abuse 
treatment services should be measured by the criterion of what services do to improve the well¬ 
being of patients, with a primary assumption that those receiving help are themselves active 
participants in the helping. 

If an agency accounts only for the costs and staff time involved in services, the patient tends to 
be treated as a case. However, if greater attention is paid to the changes in the condition of the 
person (the patient) as a result of services provided (the outcomes), then a fundamentally 
important shift has occurred. It is a shift from thinking about a patient as being acted upon by an 
outside agent to thinking of the patient as being an individual who is changed in part because of 
that patient's own actions. This shift has direct ethical content. 

Beyond the renewed emphasis upon the needs of the patient in outcomes-based accountability, a 
second difficult set of ethical issues arises in detennining what is fair to measure in holding an 
agency, a program, or even a staff person accountable for achieving certain specified outcome 
standards in treatment. The concept of fair measures has been used to underscore the importance 
of the patient's and staff person’s perception that what is being measured is an appropriate 
indicator of what progress has (or has not) been made. 







Fairness must have both process and substantive meaning. Procedurally, the staff or patients 
whose behavior or performance is being monitored need to feel they had a role in setting the 
outcomes, rather than feeling that the outcomes were forced upon them. To be sure, staff have to 
take a mandated standard seriously, but outcomes-based accountability clearly works best if the 
behavior being monitored is voluntary and the standard is seen as fair. 

Substantively, fairness must be gauged in terms of external effects that may make the outcomes 
achievable or realistic. If unrealistic measures are set, staff will not take the outcomes seriously 
because they know they will not be able to ensure that patients achieve them. These issues of 
external impact will be discussed at greater length below. 

Several other ethical issues arise in addressing outcomes: 

• Who has the authority and legitimacy to define outcomes or outcomes standards? How do 
different perspectives of different stakeholders affect the definition of the goals and needs that 
underlie selection of outcomes? 

• Who negotiates which outcomes indicators will be used as measures of performance? 

• Who decides if the measures are fair? 

• Who monitors to determine if the measures are achieved, and from what set of motives? How 
independent must that monitoring be from having an interest in the agency's or the program's 
survival? 

• How are decisions made to reflect outcomes in funding determinations? Can outcomes be used 
to shift funding from programs with poorer outcomes to those with better outcomes? What are 
the ethical effects of closing down programs or shifting funding? What are the ethical effects of 
ignoring programs' effectiveness and not shifting funding to programs with better outcomes? 

• How are external effects on outcomes taken into account? 

• What is the legitimate role of treatment providers and the patients who are the intended 
beneficiaries? How much of a contribution is it fair to ask treatment staff to make? What 
responsibilities do patients have for improving their own well-being? 

• What are the rights and responsibilities of funders, including taxpayers, in determining the 
outcomes by which to judge whether or not programs are successful? 

• How do we assure that outcomes affecting people are given at least equal weight to those 
affecting dollars; in other words, how can patient outcomes be given standing in the face of the 
great importance given to fiscal outcomes 

• How are the different groups affected by outcomes best taken into account in the process of 
defining and monitoring outcomes? 

These are questions about power, fairness, and the willingness to make large changes in current 
ways of doing business. Each of these questions raises further questions of equity and 
effectiveness, and the answers to them are affected by the different perspectives from which each 
stakeholder judges equity and effectiveness. 

Unfortunately, outcomes monitoring is often approached as though it were a far more technical 
process than it is. We look to researchers to develop the "clean data" needed for an objective 
assessment of whether programs are succeeding in their precisely described missions. Outcomes 
monitoring would probably be easier if it were as technical, impersonal, and dehumanized as it is 
often assumed to be—but reality does not present itself quite so neatly. 



Outcomes monitoring is a way of setting and following up on goals, and the process of goal 
setting cannot be conducted as a technical exercise. Values choices come into play repeatedly in 
community or agency goal setting, and it is impossible not to take these values issues into 
account. To assign priority to one value over another, when both may involve the lives of 
children, families, or patients needing services, is to make choices among different goods. 
Sometimes these are very difficult choices in which each choice involves strongly held values 
that are in conflict. No matrix, gaps analysis, or needs assessment will make clear which values 
should be given priority in any given system or community. 

So the first ethical task in working with outcomes is to be clear that choices about priorities 
involve ethics and not to try to obscure that reality with technical or analytical verbiage. 


Choosing Target Groups for AOD Services 

Very difficult questions arise in human services systems in making the set of choices that must 
be made among many different categories of patients who need various kinds of help. Sometimes 
the human services system seems to be trying hard to spread its resources widely across many 
different groups in need, rather than concentrating resources in ways that might result in denying 
any one group access to services. This allocation may lead to resources that are provided in 
"dosages" far too small to be effective. Providing many patient groups with token-level, 
symbolic allocations of resources may be politically effective—but it is not often 
programmatically effective. 


Ethical Issues and 
Outcomes-Based Accountability 

• Who defines the outcomes standards? 

• Who decides if the measures are fair? 

• Who monitors whether measures are achieved? 

• How much responsibility do patients have for improving their own well-being? 

• What are the rights and responsibilities of funders in determining whether programs are 
successful? 

• How do we weigh the importance of patient need as opposed to fiscal outcomes? 


A thorough approach to outcomes monitoring, as argued elsewhere (Young et ah, 1993), requires 
decisions about priorities. Often, human services systems allocate resources by earmarking a 
portion of funding for high-priority groups, based upon a national or local sense of the crisis 
affecting those groups. Thus, some drug treatment constituency groups have demanded a set- 
aside of funds for pregnant users, HIV-positive patients, and other patients with certain 
conditions deemed more urgent than others. 








There are ethical consequences of setting priorities when there are more patients than available 
resources can help. The first ethical issue is identifying the values underlying the establishment 
of priorities and assessing the results of those choices. If political lobbying by the advocates for 
one group is successful, does that success come at the expense of other groups with needs which 
are arguably as great or greater? Advocates have every right to seek the best possible treatment 
for "their" patients, but if there is no standard other than what works best politically, power 
becomes the only criterion for decisionmaking. 

Another set of ethical concerns arises in the question of the roles patients themselves have in 
setting priorities and in designing programs that flow from those priorities. If patients do not 
have a role in the debates over resources, a critic may be justified in labeling such a system 
agency-centered rather than patient-centered. Though the focus on outcomes may force a greater 
emphasis upon patients, there is no guarantee that, simply because large amounts of data are 
being compiled about patients, patients themselves will contribute to making the decisions that 
flow from the collection of that data. 

Ethical concerns cannot answer these questions—but they can help in posing them as a part of 
the decisionmaking process about how scarce resources are allocated. 


Choosing Outcome Measures: Which Kinds? 

The process of selecting outcomes as goals to guide agencies' work requires making some 
important tradeoffs that have ethical content. One of the most important of these is the tradeoff 
between cost effectiveness and treatment effectiveness. The former is funder driven, while the 
latter is patient driven. Both funders and patients have rights (and responsibilities), but often the 
cost-effectiveness issues related to fiscal outcomes take precedence over patient outcomes. 


The process of selecting outcomes as goals to guide agencies' work requires making some important 
tradeoffs that have ethical content. One of the most important of these is the tradeoff between cost 
effectiveness and treatment effectiveness. The former is funder driven, while the latter is patient 
driven. Both funders and patients have rights (and responsibilities), but often the cost-effectiveness 
issues related to fiscal outcomes take precedence over patient outcomes. 


In the long run, this emphasis may lead to choices that are not only ethically weak, but also 
ultimately fiscally and programmatically weak. For example, if a short-run savings is achieved 
by selecting patients who may need less help than more difficult patients, two results may follow 
that can jeopardize both fiscal and patient outcomes. First, the fiscal outcomes may eventually be 
worse because the harder-to-serve, chemically dependent patient who is left untreated will 
continue to require aid from a number of systems (for example, criminal justice or social 
services) at much higher short- and long-range costs. Second, the patient effectiveness of such 







short-range choices may also be limited, since treatment will be restricted to those methods that 
work only for the "easier" patients. As a result, there will be little pressure on agencies to 
improve their performance to help the patients who need help most. 

This issue of "creaming"—the effect of performance-based contracting in creating incentives for 
agencies to select the patients easiest to serve—has been treated extensively in the literature on 
outcomes. If patients with greater needs receive lower priority in an outcomes-based system, the 
accountability this instills will be destructive of public policy goals in some important respects. 
Ethically, the use of outcomes requires some consideration of the incentives created by 
outcomes-based funding and the impact of those incentives on different patients. 

In a somewhat different context, discriminating against patients who may have the same claims 
to public services but differ in their severity of abuse may be justified based on some screening 
of patients' willingness to work on their own behalf. Typically, however, it is the agency's 
assessment of the patient that determines the priority given, rather than a fair test of the patient's 
willingness to assist in treatment. 

The use of outcomes measures as a basis for funding raises another question about fairness: How 
fair is it to base all or part of an agency's funding upon its performance? The question, in part, 
concerns external factors that may have a great deal to do with whether the agency can achieve 
its goals. In the AOD treatment field, an example would be expecting a program to succeed 
where length of stay was limited to 1 week. Therefore, it is important to evaluate outcomes 
measures against the capacity of the agency to achieve certain outcomes in the context of local 
setting and conditions. 


The use of outcomes measures as a basis for funding raises another question about fairness: How fair 
is it to base all or part of an agency's funding upon its performance? 


Capacity also involves the self-sufficiency of the agency. If funds for a treatment program 
serving teenaged parents are linked in part to its success in reenrolling teen parents in school, but 
the school system has restrictive policies that make it very difficult for teen parents to attend, the 
treatment provider may be penalized because of policies it does not control. Similarly, if the 
treatment provider needs childcare and transportation resources to achieve its programmatic 
goals but controls neither, it is neither sensible nor fair to penalize the treatment provider for not 
achieving results that are beyond its control. 

The process of assessing whether the measures are fair can rely in part upon the ethical concept 
of the "neutral observer," but it also has to take into account the subject-object distinction 
explained earlier. A disinterested observer can argue that a 75 percent recovery rate, measured in 
an objective, verifiable way, is "fair." But from the points of view of the staff whose performance 
is being judged—and the patients whose lives are even more fully involved—the fairness of the 
measures may be perceived differently. Both the disinterest and determination of patients and 







staff who will be affected by outcomes have ethical meaning, and both need to be considered in 
assessing whether outcomes are fair measures. 


Ethics of Reallocating Funds 

The ethics of the effectiveness of treatment raises additional issues. If programs are ineffective 
and yet scarce resources are spent on them, what are the obligations of systems to redirect 
resources toward programs that really help people? 

Are results being taken seriously if it is known that a treatment program or method does not 
achieve the outcomes it is being funded to achieve—and nothing is being done to reallocate 
resources away from such an ineffective program or method? More specifically, if patients are 
on waiting lists for programs that have been proven effective and funds are being allocated to 
programs that have been proved ineffective, is there a moral responsibility to seek to reallocate 
funds toward the programs known to be effective? 

If outcomes monitoring is intended to improve resource allocation and accountability for 
effective use of funds, it is necessary to make the hard choices that reallocation demands. While 
funded agencies and programs may argue against reallocation proposals, their arguments must be 
weighed against other uses of the funds which may shift resources to programs of more proven 
value to patients. Whether a system is ultimately patient centered or agency centered is revealed 
in part by whether funds are ever redirected to programs that have proven more effective in 
serving patients, as contrasted with meeting the needs of funded agencies operating programs 
that have not proven their effectiveness. 


If outcomes monitoring is intended to improve resource allocation and accountability for effective use 
of funds, it is necessary to make the hard choices that reallocation demands. 


Merely setting up a new management information system to monitor outcomes will not ensure 
that appropriate resource reallocation occurs. Human services agencies have collected massive 
amounts of data for decades, through both manual and electronic means, without causing any 
appreciable impact on shifting resources from ineffective programs to effective programs. A 
sense of urgency—and a healthy skepticism about current claims of effectiveness—are required 
to undertake the difficult, often controversial task of redirecting funds. Ethical content arises in 
comparing the claims of currently funded agencies and programs against those that are 
underfunded or unfunded. But once knowledge of ineffectiveness is in the hands of 
decisionmakers, issues of the misallocation of resources become directly policy relevant. If 
planners and agency leaders are not aware of this dimension of information system building—or 
do not want to involve themselves in the tangled thickets of reallocation—the use of outcomes 
data will be limited to operational adjustments that will often end up as little more than tinkering 
at the margins of ineffective programs. In times of scarce resources, such tinkering is at best 







ethically questionable and may represent serious misallocation of vitally needed resources for 
patients. 

Information in Chapter 8 about turning the empirical results of outcomes monitoring into policy, 
including fiscal policy changes, should be considered in the context of this discussion of ethics. 


Ethics of Using Outcomes in AOD Treatment 

Additional hotly debated ethical issues in the field of substance abuse treatment go to the heart of 
public and personal perceptions of addiction and recovery. Many individuals' beliefs about 
addiction have grown out of personal experience and observations, and ethical judgments are 
likely to follow from these perceptions and biases. A central issue is the responsibility of the 
addicted person for his or her own life and whether or not the causes of the disease of addiction 
is more the fault of the patient than any other disease might be. Other ethical issues arise around 
whether or not society is ethically entitled to insist that the addicted person seek treatment and 
whether or not society is ethically required to provide a treatment opportunity for every addicted 
person who seeks to go into rehabilitation and recovery. Another ethical issue is the perceived 
difference between legal and illegal substances, which can lead to very different value judgments 
about patients. 

Further ethical issues in treatment arise from the different resources provided for different 
patients, based on personal ability to pay for services or differing reimbursement systems. Thus, 
addicted celebrities and star athletes have access to the best treatment, while an unemployed 
individual with the same condition is treated very differently and may even have to meet 
eligibility criteria to qualify for treatment. Waiting lists for some programs require callbacks to 
remain on the list; in contrast, some private hospitals will send cabs to make it easier for people 
to enroll. This disparity may result in very different outcomes, and ignoring these issues will 
obscure important problems in AOD treatment access. 


Ethical issues in treatment arise from the different resources provided for different patients, based on 
personal ability to pay for services or differing reimbursement systems. This disparity may result in 
very different outcomes, and ignoring these issues will obscure important problems in AOD treatment 
access. 


Other issues specific to the AOD system include the fairness of the expectations society places 
upon patients in defining when they are "well." Should the expectation (and thus the measured 
outcomes) be that no more AOD abuse will take place; that less use will take place; that 
abstinence will follow treatment; and that society will benefit as a result of less crime, lower 
health costs, and increased worker productivity? What is fair to expect? Different systems and 
agencies draw these lines in different places. An employee assistance program (EAP) cares about 







workers' performance and productivity, a treatment program for homeless people wants to ensure 
stable housing, and the child welfare system wants to know if parents are ready to be responsible 
for their children. This measurement of effectiveness of treatment in fundamentally different 
ways results in a loss of "horizontal equity"—the social policy criterion that people in similar 
situations should be treated the same. 

What is apparent is the enduring power of the categorical funding system to reduce the patient to 
"pieces of a person"—to whatever eligible definition of need will trigger funding, rather than 
considering the patient as a whole person in need of combined services across systems. Some 
portion of the AOD outcomes issue, then, demands a point of view that incorporates integrated 
services; the categorical system should not be allowed to fragment treatment into whatever the 
treating agency defines as its mission—rather than what patients (and their children or families) 
want and need. 


Applying Ethical Ideas 

There is no ethical calculus that can easily be applied to determine whether a decision is ethically 
valid. Rather, there are ethical checkpoints that may be used to frame the issues for a thoughtful 
analysis of the moral dimension of policy choices, which can be added to the social, economic, 
cultural, and political dimensions. Some of these checkpoints are: 

• Is this choice fair to those involved? How would they respond to that question? How would a 
neutral observer respond? 

• Has the impact of this choice on all parties been weighed carefully? These parties include 
providers, patients, and the community. 

• Is there reciprocity in the responsibility for the choice; that is, have both those providing help 
and those receiving help accepted obligations to make it effective, according to what they are 
able to do? 

• Have those with the least power to raise issues and implement policy around those issues been 
given help in that task by those with more power? 

• Are decisionmakers driven by motives that are narrow and largely self-interested? 

• What is known about the results of previous policy choices similar to this one, and how did 
those choices affect their intended beneficiaries? 

• Is there evidence that decisionmakers have or sought to have empathy with those most affected 
by their decisions? 

• Have the values underlying policies been identified and debated openly as values issues, or are 
they concealed beneath the economic, political, or technical issues? 

• Has an attempt been made to identify the interests of the whole community—the values that 
bind it together—as well as the more special interests that may seem to conflict with the policy 
choice? 

• Has there been an attempt to understand the full context of the family setting in which 
treatment is being provided? Have the needs and strengths of children, parents, and other 
family members been added to the assessment of what the patient needs? 

• Is there an institutionalized, independent channel through which patients can regularly evaluate 
the programs that serve them? 



Endnote 


l.This chapter was written for the consensus panel by Sidney L. Gardner, M.P.A., M.A. 


Chapter 4—Outcomes Monitoring Methods 

This chapter discusses the methods and principles of designing instruments, gathering data, and 
sampling from patient populations and programs for an outcomes monitoring system (OMS). In 
the next chapter of this Treatment Improvement Protocol (TIP), the data needs of an OMS are 
described, and suggestions are given to help the director and staff of a single State agency (SSA) 
decide which data are relevant for the agency's specific monitoring purposes. 

On the next page is a content outline for the chapter to guide the reader. 


General Principles 

As OMSs are designed, decisions have to be made about how many patients and programs to 
monitor, which questions to include, which instruments to use, and when and how often to 
follow up patients. Conflict can arise between advocates of scholarly research design and 
advocates of a more pragmatic approach. SSAs, treatment providers, and other stakeholders must 
weigh alternatives carefully when deciding what system will best fit their needs. Chapter 2 
provides suggestions for considering a variety of perspectives while reaching consensus. 

The fundamental principles of OMS design are feasibility, applicability, and utility. Since these 
principles can be used to guide all OMS-related decisions, they are briefly explained. 

Feasibility 

Feasibility refers to the extent to which planned actions can actually be carried out. OMS 
planners must take into account the demands of the system on State agency resources, treatment 
providers, and patients. Whatever the amount of funding available to support the OMS, these 
resources will have to be allocated carefully. Putting all the money into instrument design will 
leave nothing for equally important aspects of the OMS, such as data analysis and report writing. 
Using instruments that require a great deal of time for treatment program staff and patients to 
complete will detract from time needed for other clinical responsibilities and patient needs and 
will build resentment toward the project. 

Applicability 

Applicability refers to the extent to which the OMS is designed to fit local needs. Since States 
differ in their treatment service delivery system structures and programs, as well as in the 
treatment populations they serve, their OMSs will differ in specific details. Designers of the 
OMS will need to take the specific characteristics of their State and programs into account. 


Utility 



An OMS must be useful; it must serve some worthwhile end. Data collection for its own sake 
cannot be justified. Utility refers to the usefulness of an OMS. The purpose of each question and 
each procedure must be clear in all participating programs. If participants cannot be convinced of 
the utility of the OMS, it will fail. Chapters 1 and 2 address broad goals for an OMS and the 
process for achieving support for the system; Chapter 8 discusses putting the findings to practical 
use. 


Content Outline of This Chapter 


General Principles 

Feasibility 

Applicability 

Utility 

Fundamentals of OMS Design 

Scientific rigor 
Standardization 

Validity and reliability of instruments 

Research Designs 

Experimental 
Posttest only 
Pretest/posttest 

Data Collection Points 

At intake 
During treatment 
At discharge or transition points 
After treatment 

Data Collection Methods 

Self-administered questionnaires 
Staff-administered interviews 
Researcher-administered interviews 
Chart review 

Biochemical alcohol and other drug (AOD) testing 







Followup Contact 

Patient consent and successful contact 
Followup methods 
Followup Intervals 
Incentives 

Gathering data from other systems 

Stages in OMS Design 

Participating patients and programs 
Incremental and hierarchical approaches 
Instrument design and selection 
Field testing 

Staff training and implementation 


Fundamentals of OMS Design 

Scientific Rigor 

Scientific rigor is the ideal to be pursued in developing an outcomes monitoring system. 
However, there are substantial costs associated with scientific rigor, and it is unlikely that single 
State agencies will be able to deploy the resources required to develop an OMS as rigorous as the 
ideal. Nevertheless, if planning teams in single State agencies concentrate on limited and specific 
service system questions, they can achieve an acceptable degree of scientific rigor without 
crippling their capacities. Ways to maintain scientific rigor while respecting budgeting 
constraints are discussed. Doing a limited number of things very well is preferable to attempting 
too much with sloppy results. Some general approaches to minimizing costs are also discussed, 
such as starting small and adding increments in design over time or collecting only minimal data 
on the universe of programs and patients and comprehensive data on only a representative 
sample. Therefore, while this TIP advocates for rigorous design, practical constraints are 
acknowledged throughout. 

Standardization 

The ability to compare results across treatment programs and groups of patients is fundamental 
to the proper design of an OMS. For purposes of valid comparisons, the standardization of data 
collection elements and procedures is recommended for all programs and all localities within the 
State. Beyond State boundaries, national standardization of data elements would expand the 
capacity to compare outcomes in different parts of the country where there is a great deal of 
diversity in the provision of AOD services. 







Standardiztion, or the ability to compare results across treatment programs and groups of patients, is 
fundamental to the proper design of an OMS. 


Standardization of data elements, or variables, means consistency in these elements, whatever 
the variable sources. For OMS purposes, the data to be standardized include patient predictor 
variables, baseline and outcome measures, and treatment variables, as described in Chapter 5. 
While standardization of data elements does not necessarily mean that all the same data must be 
collected at every program on every patient, it does mean that when information is collected on 
specific data elements, those data elements are defined clearly and consistently and contain the 
same response choices. 

For example, suppose it is agreed that frequency of alcohol use is a required data element. A 
variety of different variables has been used in existing instruments. Suppose one program were 
to use a variable that classified alcohol use as daily, at least once a week, at least once a month, 
less than once a month, or never; and another program were to use a variable that asked for the 
number of days in the previous 30 in which alcohol was consumed. While disagreement may 
exist as to which variable is preferable, the advantages of standardization are obvious. Unless all 
participants in the system use the same variable, patient responses cannot be meaningfully 
compared across programs. Standardization also ensures that similar treatment services, defined 
consistently, are similarly coded. 

The need for standardization also applies to data collection procedures whenever possible. For 
instance, if certain infonnation is to be elicited through an interview with the patient, this method 
should be consistently used in all the programs participating in the OMS. If inpatients are to be 
interviewed within 7 days of admission but only after the effects of recent alcohol and/or drug 
ingestion have cleared, this timeline must be adhered to consistently in all programs where data 
are being collected. If treatment service data are to be recorded by a primary counselor in one 
program, primary counselors should be data recorders in all programs. 

In some parts of the country, a great deal of progress has been made toward standardization. The 
Federal Client Minimum Data Set is a standardized set of commonly used demographic and 
other intake variables. Because this data set is mandated for providers receiving Federal block 
grant or other State agency funds for treatment, it is described in detail in the next chapter (see 
Exhibit 5-2). This client data set is recommended as the foundation for the State OMS. 

Standardization of patient assessment variables is exemplified by instruments such as the 
Addiction Severity Index (ASI) (Longabaugh, 1991), also discussed in the next chapter. Cost- 
conscious planners and designers of an OMS will build on existing data collection strategies. 
Integration of standardized data elements across levels of government and other agencies with 
oversight or evaluation responsibilities will also help eliminate extra work for provider agencies, 
which in many cases report data in different formats for local, State, and Federal agencies. 







While the benefits of standardization are clear in terms of producing higher quality data and 
making valid comparisons possible, there may be some objections to standardization. Programs 
that focus on special issues or serve special populations may find that standard descriptors do not 
provide sufficient information about the unique needs of their patients or aspects of their 
programs. Flexibility must be considered to respond to special needs, but it can be offered as an 
adjunct to standardization rather than as a substitute for standardization. For instance, 
standardization could be limited to a core set of variables required for patients in all programs. 

At the discretion of the individual agency, a supplemental set of variables could be added to 
address questions of greatest interest to the special programs and populations. The advantage of 
such a flexible plan is that it would provide comparable data for systemwide decisions related, 
for example, to resource allocation, while still allowing specialized programs to address their 
own particular needs. Adaptations of the Federal Client Minimum Data Set and the Addiction 
Severity Index have been incorporated by some States into their outcomes monitoring systems. 
These OMSs provide examples of modifications to meet local needs and are illustrated in 
Chapter 5 and Appendix B. 

The balance between standardization and program flexibility should be addressed during the 
OMS planning process. Balancing the needs of various stakeholders is one reason it is essential 
to have their involvement at the earliest stages of planning (see Chapter 2). Respecting the needs 
and wishes of various groups will enhance the OMS and the buy-in from providers. With 
sufficient flexibility built into the system to meet their needs, provider agencies are likely to 
respond positively and appreciate the potential of the OMS to address State policy questions and 
their own service evaluation objectives. 

While standardizing core elements from all data domains is an ideal rather than a reality for most 
systems, the vision of integrating elements within the OMS to meet local, State, and Federal 
reporting mandates should guide the process. In practice, all systems are developed iteratively, 
and the development of an OMS is also likely to be an evolving, one-step-at-a-time process in 
most States. Standardization across systems or throughout different levels of government, 
therefore, will probably arise as an outcome of systems development iterations. 

Validity and Reliability of Instruments 

Validity and reliability are close cousins to standardization. Standardization is one method of 
achieving validity and reliability. Validity refers to usefulness, and reliability refers to accuracy. 
Validity means that an instrument measures what it purports to measure (this is technically 
known as construct validity). A valid inventory on depression among patients must measure 
depression as distinguished from anxiety disorders or grief reactions. 


Validity means that an instrument measures what it purports to measure. 


A reliable instrument is an accurate instrument; it provides repeatable, consistent results. 
Reliability means that the question or instrument will elicit the same answer from the same 







respondent regardless of the interview conditions or other factors (except an actual change in the 
respondent's condition). For example, if a question about physical abuse is vague and abuse is 
not defined, the respondent might answer "yes" to the question at one point and "no" at another; 
this question would not produce reliable results. 

Consistency of responses at different points in time is known as test-retest reliability. 
Consistency of responses to different interviewers asking the same questions or consistency of 
observer ratings of patient behavior, is known as interrater reliability. 


Reliability means that the question or instrument will elicit the same answer from the same 
respondent regardless of the interview conditions or other factors. 


Standard procedures exist to design questionnaires and structured interviews to ensure validity 
and reliability. These procedures include pilot testing the instruments and conducting a series of 
data analyses on the responses, followed by refining the instruments and repeating the process. 

Instrument design is a science in itself, and sound instruments are expensive to develop and test. 
While the level of rigor of the OMS design will depend on available resources, measures and 
instruments used must have the highest levels of validity and reliability possible. Results from 
instruments that have gained wide acceptance in the research community will be less open to 
challenge than those from untested instruments. 

External validity is another important consideration; external validity is commonly called 
generalizability. Generalizability refers to the extent to which an instrument that has been found 
valid for one group is valid for other groups as well. For example, an instrument developed for 
use with adult white males in the United States, as many AOD-related instruments have been, 
cannot be assumed to be valid for adolescents, females, persons of color, or persons from other 
countries or cultures. For an instrument to be recommended for widespread use, it must have 
undergone testing on a wide variety of populations. State OMS planners must keep this principle 
in mind as they select instruments, particularly when treatment populations of interest include 
special subgroups such as American Indians, Southeast Asian or Mexican immigrants, pregnant 
women, people with mental illness, or people with little formal education, to name a few. Many 
of these populations have not been included in development of existing instruments; at best these 
instruments would need to be pilot tested with samples of these groups or, at worst, used with 
great caution. 


Generalizability refers to the extent to which an instrument that has been found valid for one group is 
valid for other groups as well. 















Principles of generalizability also apply to study findings as a whole. A well-designed study may 
find that a specific program with a unique array of services produced excellent outcomes for its 
patients. A careful review of the study design and data analyses might find the results to be 
internally valid. However, these results are not necessarily generalizable to other groups of 
patients with different characteristics, histories, and needs. To ensure generalizability, the same 
program would have to be delivered to a wide spectrum of different patient groups in different 
locations at different times. 


Research Designs 

Experimental Design 

As noted in Chapter 1, the strongest type of research design is the experimental design because 
of its capacity for demonstrating causal relationships between interventions and outcomes. In an 
experimental design, patients are randomly assigned to two or more groups. One group receives 
the conventional treatment, while the other receives the experimental treatment or no treatment at 
all. (Withholding or delaying treatment poses serious ethical problems and thus is not considered 
an option for AOD treatment studies.) While random assignment can be an extremely effective 
tool for research purposes, it has its own limitations, which are often overlooked. Patients must 
grant infonned consent for random assignment to alternative treatments; those willing to be part 
of such an experiment may not necessarily be representative of the group as a whole (for 
instance, they may be more compliant or more altruistic), thus producing a biased sample. 

Experimental designs are not appropriate for an OMS; by definition, an OMS records findings 
from the system as is; it does not manipulate the treatment system. This is not to say there is not 
a place for experimental design in treatment research, however. Such projects could be 
developed and applied on a smaller scale to address questions raised by OMS results or not 
covered by the OMS. 

Posttest-Only Design 

A posttest-only design is the simplest to develop and implement. Data are collected from patients 
at some point following treatment and then analyzed to determine if certain groups of patients 
have had better outcomes than other groups receiving the same services. This design can provide 
accurate information on the status of patients following the intervention, for example, if they are 
currently using AODs, if they are employed, or if they are physically healthy. A posttest-only 
design cannot, however, provide information on whether patients improved since treatment. 
Without baseline measures for comparison, most outcome measures are meaningless (Alio et ah, 
1988; Sobell et ah, 1987). The exceptions are those that have no meaningful baseline, such as 
treatment satisfaction ratings or the use of community recovery resources after treatment. 


Pretest/Posttest Design 


A pretest/posttest design is recommended for an OMS because it balances scientific rigor with 
practicality. A pretest/posttest design allows for the measure of change over a period of time. 
"Pretest" and "posttest" are analogous to "before" and "after" or "baseline" and "outcome." 
"Pretest" and "posttest" are generally used to refer to overall study or system design and the 
process of collecting comparable measures before and after the intervention or treatment. The 
terms "baseline" and "outcome" are typically used to refer to the measures or variables 
themselves. 

The limitation of pretest/posttest designs is that they do not prove a causal relationship between 
patient outcomes and treatment. Even when significant improvements can be documented, the 
possibility exists that factors other than treatment could account for the changes. Changes in 
patients' behavior or functioning may have occurred even without the intervention, for example, 
as a result of community changes in law enforcement efforts, family crises, influences of friends, 
change in employment status or job satisfaction, maturation (especially in the case of 
adolescents), or a host of other potential influences. Unless the effects of other factors can be 
ruled out, or controlled for in statistical analyses, cause-and-effect relationships cannot be 
proven. 

Pretest/posttest designs can provide satisfactory results when sample sizes are large enough and 
populations sufficiently diverse that comparison groups can be derived from the database. 
Patients can be matched on many characteristics and factors so that differences found can 
reasonably be assumed to be related to differences in the services received (California 
Department of Alcohol and Drug Programs, 1994). 


Three types of research design are: 

• Experimental 

• Posttest only 

• Pretest/posttest. 


Data Collection Points 

An OMS should incorporate data collection at four periods in time: 

• At intake 

• During treatment 

• At discharge or other transition points 

• After treatment. 


At Intake 







A patient intake form should collect basic data on every patient in the system. The Federal Client 
Minimum Data Set, for example, can easily be incorporated into an OMS intake form. (Chapter 5 
addresses this point in greater detail.) Intake items are generally factual and straightforward. The 
advantage of collecting at least minimal information immediately is that even if the patient 
leaves treatment soon after starting, descriptive information will be available. 

During Treatment 

Two kinds of data should be gathered during treatment: patient data and treatment service data. 
The patient data will be more comprehensive than that collected at intake. It is recommended that 
more detailed historical data and that which requires patients' evaluation of their problems be 
delayed until the initial effects of AOD ingestion have cleared and the patient is not in acute 
distress. If programs are asked to wait a week or so to collect a certain set of patient data, it will 
be lost for those patients who drop out of treatment early in the process. Nonetheless, the 
likelihood of increased patient cooperation and increased accuracy of infonnation make this 
timing preferable. 

Treatment service data can be collected at various points during the treatment process. The next 
chapter describes two instruments that can be used to collect a weekly record of services 
received by each patient and discusses the variety of treatment information that may be useful to 
collect. Appendix B also illustrates how some States have incorporated treatment service data 
into their OMSs. 

At Discharge or Transition Points 

A patient discharge form should include at minimum the date of discharge and discharge status 
(treatment completed, left against staff advice, etc.). A list of discharge referrals could also be 
included. A discharge form could also include service information, such as the type of services 
received and the duration of treatment (total days for inpatients or total hours for outpatients). 
Costs of treatment might also be recorded here. This form should be completed on the day of 
discharge to ensure accuracy. 

Discharge from an inpatient or residential setting may be the end of a treatment episode. It has 
become more common, however, for inpatient treatment to be followed by outpatient treatment, 
sometimes called a "step-down" model. In some situations, patients who begin treatment in 
outpatient settings may need to be transferred to inpatient or residential treatment. For 
administrative purposes and recordkeeping, transfers to different settings are often treated as 
discrete admissions and discharges even when they are part of the same treatment episode. 
Whether or not the change in setting or intensity of services is considered a formal discharge, 
these transition points are an ideal time to collect the same kind of information recommended for 
inclusion in a discharge fonn. 

AOD treatment may well evolve in the future into a continuing care model in which a patient 
receives care on an "as needed" basis, as in the case of many other illnesses. Even now, many 
programs offer continuing care (for instance, a session per week) for months or even a year or 
longer after completion of a formal treatment episode. In Minnesota, where this continuing care 



is common, the end of the more intense phase of treatment (typically 1 or 2 months) is treated as 
the "discharge" point for purposes of outcomes monitoring, and attendance at continuing care 
sessions is recorded during the 6-month followup interview. While the concept of "treatment 
completion" has clear benefits for ease of data collection and analysis, changing clinical realities 
may blur this concept in the future, a possibility that must be taken into account in the design of 
OMSs. 


While the concept of "treatment completion" has clear benefits for ease of data collection and 
analysis, changing clinical realities may blur this concept in the future, a possibility that must be taken 
into account in the design of OMSs. 


After Treatment 

The essence of an OMS is outcomes information. While patient changes during treatment can be 
considered preliminary outcomes (Manu et ah, 1994; Wickizer et ah, 1994), posttreatment data 
are necessary to measure sustained change. Because followup data collection is more difficult 
than intreatment data collection, it will be addressed separately later in this chapter. 


Data Collection Methods 

Data to be included in an OMS can come from a variety of sources and can be gathered by one or 
a combination of several techniques. The primary source will be the patient. Other possible 
sources include spouses, partners, significant others, other family members, treatment providers, 
and other official records. In designing the system, the possibility of multiple information 
sources and multiple data-gathering techniques should be considered. 

There are several different methods of getting data from these sources. Best practice in the field 
may be a mix of several of these methods, and the best mix will be determined by system 
resources and goals. The methods of obtaining data include the following: 

• Self-administered questionnaires 

• Staff-administered interviews 

• Researcher-administered interviews 

• Chart review 

• Biochemical AOD testing. 

Self-Administered Questionnaires 

Some instruments are completed by patients without the interaction of a staff member. These 
self-administered questionnaires offer the advantages of economy and efficiency. They can 








sometimes be administered to a group, and administration can be arranged at the patient's 
convenience rather than the provider's or program staffs convenience. Self-administered 
questionnaires can also be completed by family members or significant others to obtain a 
collateral perspective on patient functioning and behavior. 

There are disadvantages to self-administered questionnaires. They may require a level of literacy 
that some patients or collateral informants do not have; if program staff are not available to 
answer questions, the questionnaires may be incomplete or inaccurately filled out. Some patients 
may need prompting and encouragement to complete forms and may fail to do so without staff 
support. 

These problems can be addressed to a certain extent by requiring that a staff member or research 
assistant review the questionnaire with the patient after completion. This help provides an 
environment of supervised administration and will help ensure that complete and usable data are 
obtained from each questionnaire. However, if this assistance requires a great deal of staff time, 
the advantages of self-administration are negated. 

Questionnaires mailed to patients for followup purposes fall into this category. Mail surveys are 
inexpensive and make minimal use of project staff time. However, they have a number of 
significant drawbacks. Mailing a survey form to a patient's residence may compromise his or her 
confidentiality (see Chapter 6 for a discussion of methods used to protect against this risk). There 
is also no way of verifying who actually fills out the survey and no way to detennine how 
thoughtfully or carefully the questions have been answered. 

Mail surveys often have poor response rates because they depend upon the patient to initiate the 
filling out and returning of the questionnaire, an action that even optimally functioning people 
are often unlikely to perfonn. In general, because of low response rates and problems regarding 
quality assurance, mail surveys are not recommended. Followup methods will be discussed in 
more detail later in this chapter. 

Staff-Administered Interviews 

Staff-administered interviews offer a number of advantages over self-administered 
questionnaires. Interviews share the burden of work between staff and patient. The interaction 
between patient and staff is an important component of this method of data collection. When an 
interviewer establishes good rapport with a patient, the likelihood is increased that the patient 
will respond truthfully. The data collector can probe certain responses for clarification or 
expansion and can also verify the patient’s understanding of the question's intent. Questionnaire 
content is not limited by literacy levels of the patient population. Staff can receive prior training 
in accurately coding responses so that errors are likely to be fewer than with self-administered 
questionnaires. The most respected interviews in the AOD field now are staff-administered 
structured interviews such as the Addiction Severity Index. 

The need for staff training may be considered a drawback of staff-administered interviews 
(Longabaugh, 1991). Staff training can be costly and, depending on staff turnover and the 
complexity of the interview, may require constant updating. Although the use of structured 



interviews may demand a fair amount of staff time, if the data collected are valid and/or used for 
multiple purposes, the end results may justify the investment in staff time. 


The most respected interviews in the AOD field now are staff-administered structured interviews. 


Researcher-Administered Interviews 

Another alternative is to have in-treatment interviews conducted by a researcher assigned to the 
OMS project rather than by treatment program staff. The major obstacle to this type of interview 
is cost. Most States would not consider this a feasible option. However, using research staff to 
conduct followup interviews has major advantages. This use of staff is discussed in the section 
devoted to followup later in this chapter. 

Chart Reviews 

Patient charts represent a potentially rich source of information about clinical assessments, 
treatment services, and patient responses to these services. Chart review is traditional in clinical 
practice, a concept that is well understood and accepted by most treatment providers. Advantages 
of using chart reviews to gather data for an OMS include the fact that chart reviews do not 
require input from the patient, and they can be performed at the convenience of the reviewer/data 
collector. 


Patient charts represent a potentially rich source of data. However, chart review is labor intensive and 
costly, and information in charts is usually not standardized. 


However, relying on chart review has some disadvantages. Reviewing patient files is labor 
intensive and, therefore, costly. The subjective and qualitative nature of most of the information 
in charts may lead to inconsistent coding. Charts also do not provide the same information for 
each patient, leading to missing or ambiguous data for some patients. For an OMS to most 
effectively use data from chart review, it is recommended that a standard instrument be adopted 
or developed statewide to document patient characteristics and treatment services. 

An alternative to reviewing charts to obtain a description of treatment services is to develop a 
standardized form on which to record weekly information listing type and frequency of services 
provided. This relatively simple type of recordkeeping can greatly improve comparisons between 
patients and across programs. Two such forms are described in Chapter 5. 















Extracting records from other sources can also be used as a method of data collection. Potential 
sources of information are other medical records, criminal justice records, and social service 
agency records. See Chapter 6 for information on patient confidentiality protections. 

Biochemical AOD Testing 

Biochemical alcohol and drug testing is objective, and this methodology has become widely 
accepted among AOD treatment systems. The most common tests are urine tests for drug use and 
blood alcohol concentrations for alcohol use. Hair analysis, a relatively recent (and expensive) 
technology, can reveal drug use over a longer time interval than more widely used drug tests. 
Saliva testing is also used in some regions. 

Biological testing has a high level of accuracy. Tests can verify or refute patients' self-reports 
about AOD use. They also can serve a deterrent function in keeping drug free those patients who 
kn ow they will be tested. However, most biochemical tests provide information only about very 
recent use. They are also obtrusive and, unless required as part of the treatment regimen, may be 
resisted by patients. 

Other problems with urine, blood, and other biological tests are that staff time is required to 
administer the tests and the laboratory analysis is expensive. The purpose of drug testing must be 
carefully weighed in light of its potential costs. If the purpose is primarily therapeutic (the 
deterrent effect), costs may be deemed justifiable whether or not the data are to be incorporated 
into an OMS. Some treatment programs already administer drug tests routinely. On the other 
hand, if the testing is to be instituted primarily for purposes of an OMS, its relative costs may not 
be justifiable in light of its added value to other data sources. 


Followup Contact 

Patient Consent and Successful Contact 

While patient consent is not technically required for contact after treatment, it is recommended. 
Chapter 6 addresses the legal issues related to patient consent for followup, and interviews with 
collateral sources of information. A sample consent form is also provided. 

Irrespective of the number or interval of followups, contacting patients is the key to a successful 
OMS and valid results. The fewer patients contacted, the less generalizable the results, 
particularly when contacts are skewed in favor of higher functioning, easier-to-reach patients 
(Gerson et ah, 1985; Harrison and Hoffmann, 1989; Stinchfield et ah, 1994a). 

The benefits of seeking informed consent are increased patient investment in the process and 
better information about locating the patient after treatment. Explaining the purpose of the 
posttreatment contact and the value of outcome information can boost patients' cooperation. The 
consent form should include detailed information on how to locate the patient after treatment. 
Patients can also be asked to provide names, addresses, and telephone numbers of others who 


would know patients whereabouts if they moved. Patients should be reassured that calls to these 
other contact persons will not reveal that the patient was in treatment (unless, of course, patients 
expressly consent to interviews with others); these other sources will be used only to try to 
secure a new address or telephone number for the patient. Potential sources of information 
include close family members and friends, social or financial workers, probation or parole 
officers, and anyone else with whom the patient expects to keep in touch. 

Followup Methods 

While the patient is in treatment, staff-administered interviews are typically conducted in person. 
At followup, however, in-person or telephone interviews can be used. Telephone interviews are 
preferred for followup, primarily because of their relatively low cost. In-person interviews either 
require former patients to come to the interview site—a burden on them that may not elicit 
compliance without an incentive—or interviewers to visit patients' homes, a very costly method. 

Telephone interviews have been shown to produce valid results and have been used extensively 
in AOD treatment outcome studies (Hoffmann and Ninonuevo, 1994). However, telephone 
interviews are of marginal use with transient or homeless patients or others without a telephone; 
because this population is disproportionately low income and socially unstable, its members are 
at higher risk for relapse. Not obtaining outcomes information on these patients will not only bias 
results, it will neglect groups for whom treatment improvements are probably most important. 
Some studies attempt to arrange for followup interviews for these patients at a location the 
patient may visit on a regular basis: a social or financial worker's office, the office of a parole or 
probation worker, a shelter, or a similar site. 

With prior patient consent (see Chapter 6), structured interviews can also be conducted with 
patients' family members or significant others to elicit collateral information regarding the 
severity of patients' problems and level of functioning. Followup interviews are sometimes 
conducted with significant others to verify patient self-report. While reports from these collateral 
sources are frequently assumed to be more accurate than patients' self-reports, this is not always 
the case. People serving as collateral sources can also distort reality, minimize or deny problems, 
forget events, or simply be unaware of some aspects of the patient's behavior. In one large 
patient-followup registry, patients were as likely to report AOD use after treatment when 
collaterals did not as collaterals were to report it when patients did not (Hoffmann and Harrison, 
1988). 

Patient followup interviews can be conducted either by treatment program staff or research 
agency staff under contract for this purpose. Patient followup is very time consuming, much 
more so than inexperienced planners typically estimate. Because of the difficulty in locating 
patients, the need for numerous callbacks, and the length of the interview itself, a great deal of 
time should be allotted to followup. When program staff are assigned this duty, it conflicts with 
their clinical responsibilities. Even well-motivated program staff are not likely to be as 
successful as research staff who are experienced in this process and have no competing 
responsibilities (Longabaugh, 1991). 



Telephone interviews are of marginal use with transient or homeless patients or others without a 
telephone; because this population is disproportionately low income and socially unstable, its 
members are at higher risk for relapse. Not obtaining outcomes information on these patients will not 
only bias results, it will neglect groups for whom treatment improvements are probably most 
important. 


Minnesota used program staff followup for about 5 years. Programs were required to attempt 
followups for a specified number of patients and document those attempts. Contact success was 
very uneven across programs. The followup sample that resulted was heavily biased toward 
those patients with less serious problems and more social stability (Harrison, 1992). With the 
introduction of its Treatment Accountability Plan in 1993, the Minnesota State agency dropped 
the requirement that providers follow up with patients and budgeted funds for an independent 
research agency to conduct the followup interviews. 

In addition to the skills and time necessary for successful followup contact and interviewing, 
other factors argue against having treatment staff conduct followup. These are referred to as 
demand characteristics and therapist bias. 

Demand characteristics refer to those characteristics of an interviewer that may "demand" a 
certain response from the interview respondent. These characteristics may be extremely subtle 
and not even in the conscious awareness of the interviewer. But treatment staff are not neutral 
and objective with respect to treatment outcomes. They are heavily invested in good outcomes. 
The former patient knows this and may want to please rather than disappoint the interviewer. 
This attitude can lead to the patient's minimizing or denying AOD use or other problems. 

Therapist bias also results from the lack of neutrality of program staff. They may want to hear 
only the best and disregard the rest, and they may not even realize this is the case. While 
objective, behaviorally based questions can minimize these effects, they can still influence the 
interviewer's interpretation of the former patient's responses. 

Followup Intervals 

In order to compare patient functioning before and after treatment, comparisons have to be made 
over the same duration of time for different patients. The timing of posttreatment followup 
contacts is one of the major decisions confronting designers of an OMS. Typical followup 
periods are 1 month, 3 months, 6 months, 12 months, 18 months, and 24 months. While 
conducting multiple followup contacts might constitute an ideal design (Longabaugh, 1991), 
doing so would be cost prohibitive for most systems. 

Different followup intervals have different advantages and disadvantages, though they are 
relative rather than absolute. OMS designers need to weigh the relative pluses and minuses to 
make the best choices for their needs. 







At the 1-month mark, when it is probably easiest to locate patients, few are lost to followup, 
thereby minimizing the effects of attrition or contact bias. Less memory distortion and forgetting 
among patients probably occur over such a relatively short period. The drawback is that the 
interval may be too short to measure any but the most immediate treatment outcomes. The 3- 
month interval has similar strengths and weaknesses. 

The 6-month followup interval has a great deal to recommend it. The contact point at the sixth 
month is not so far removed from the time of treatment discharge that too many patients are lost. 
The interval is of sufficient length for the patient to have established some meaningful measures 
of functioning. Many treatment outcome studies have found that the vast majority of relapses 
occur during the first 6 months after treatment (McLellan et al., 1992a). Most patients who 
remain abstinent for the first 6 months are likely to continue with a sustained recovery 
(Hoffmann and Harrison, 1988). Thus, even though good arguments can be made for longer term 
followup, selecting sixth-month followup offers a reasonable compromise and has been used 
extensively in treatment outcome studies (McLellan et al., 1992a). 

Followup contacts at 12, 18, or 24 months (or even longer) have the advantage of providing 
information on long-term status posttreatment. When this approach is used, patients are typically 
contacted at several contact points, rather than the interviewer's attempting the first contact at 12 
months, for example. Contacting the patient every 6 months can protect somewhat against 
sample attrition; however, the longer the time between contact and treatment, the greater will be 
the proportion of the sample that is lost. Contact bias must be considered in interpreting long¬ 
term results. At all contact points, better functioning patients are more likely to be contacted than 
poorer functioning patients; the bias may grow in magnitude the farther from treatment the 
followup extends. 

Followups at 6 and 12 months would probably be ideal for most outcomes monitoring systems. 
These are the intervals Tennessee has chosen for its OMS. If only one followup is feasible within 
budget constraints, the 6-month contact point is probably preferable because of the higher 
contact rate associated with the shorter interval. This is the choice made for the Iowa OMS. 

Regardless of the duration of the followup interval or intervals selected, the duration of time for 
which pretreatment functioning is assessed must be the same. That is, the intake assessment must 
cover the same period of time as the interval selected for followup. For instance, if the only 
contact is at 6 months posttreatment, questions at intake must address the 6 months preceding 
treatment. If the only contact is at 12 months posttreatment, the baseline assessment should also 
be 12 months. 

Establishing a 6-month pretreatment and followup interval does not rule out using shorter 
windows for some measures. The Addiction Severity Index, for example, has some useful scales 
that are based on 30-day windows. An OMS could be designed to include both 6-month and 30- 
day measures. This is the approach taken by the Minnesota State agency (see Chapter 5 for more 
details). With this approach, one set of questions covers the 6 months before and after treatment, 
and another the 30 days before treatment and the 30 days before the 6-month followup interview. 


Incentives 



To attempt to increase contact success rates, some projects have printed postcards to be given to 
patients at discharge from treatment so that they can notify researchers of a new address; others 
provide a toll-free phone number for the same purpose. One possible though controversial 
strategy to reach a greater number of patients is the use of financial or other incentives. Incentive 
fees typically range from $2 to $25, depending on the length of the interview and whether or not 
a drug test is required. Alternative incentives to cash payments include bus tokens, food coupons, 
or other items of value to the patient. 

Gathering Data From Other Systems 

Patient pretreatment and posttreatment data can be compiled from other sources. Other systems 
may contain information on patients' use of medical care, detoxification admissions, driving 
offenses or other arrests, or public assistance. Such information can be used as primary outcome 
data or as collateral data to verify patient self-report. Chapter 6 discusses legal issues related to 
using data from collateral sources. 


Stages in OMS Design 

Participating Patients and Programs 

In designing the OMS, planners will also have to make decisions about the numbers of treatment 
programs and patients to be monitored. While it might be ideal to monitor all patients, such a 
course is probably not feasible. The next section will examine incremental and hierarchical 
designs that may achieve desired results without overburdening available resources. Hierarchical 
and incremental designs can both be part of a long-term, feasible, OMS implementation process. 

Incremental and Hierarchical Approaches 

Incremental designs are those that are phased in over time. For example, a State with no previous 
AOD OMS experience might start by introducing a patient intake fonn. After it is working well, 
a discharge form might be introduced. Eventually, a more comprehensive patient assessment tool 
or a services record might be added. Finally, posttreatment followup could be included. 

Hierarchical designs are those that collect some minimum set of data on all patients and 
programs in the treatment system and more comprehensive data on a subset. The Minnesota 
State agency has taken this approach in its OMS (see Appendix B). Intake, history, and discharge 
forms are required for every patient. This process is ongoing. The Minnesota Treatment 
Accountability Plan adds data collection components for a subset of patients at each program: a 
modified ASI and a Treatment Services Record is required. This subset of patients is also 
interviewed 6 months after treatment. This approach combines the advantages of having some 
information on all patients and programs with having sufficient information to answer the State's 
questions about the best services for different groups of patients. 


The 6-month followup interval has a great deal to recommend it. The contact point at the sixth month 
is not so far removed from the time of treatment discharge that too many patients are lost. The 
interval is of sufficient length for the patient to have established some meaningful measures of 
functioning. 


In a hierarchical approach, intensive data collection would not need to be ongoing. To minimize 
costs, data collection could be accomplished within a specified time frame and then discontinued 
while the results were analyzed and policy changes and program improvements put into effect. 
Then the data collection could be reinstituted and the process repeated. An iterative design such 
as this—maybe 1 year on and 2 years off—has some appeal if resources available for an OMS 
are greatly limited. Treatment providers would also get a break from extensive data collection 
demands. 

With any approach that does not collect the same data on all patients, some method of sampling 
must be used. Sampling is a means of studying a representative segment of a population to gain 
knowledge of the whole. In the context of outcomes monitoring, samples can be taken of 
patients, programs, or both. How extensively sampling is used will depend on a system's budget 
and human resources constraints, but it is unlikely that any system will have the resources to 
measure outcomes for the universe of patients or programs, particularly on an ongoing basis. The 
need for sampling is a pragmatic reality, and sampling will probably be used in most outcomes 
monitoring systems. 

Usually there are enough similarities among groups of programs or patients that properly 
designed samples will provide sufficient information to draw generalizable conclusions. In most 
cases, following up the universe of patients or programs will constitute an unnecessary and 
unjustifiable expense in terms of dollars and staff time. 

The most important principle of sampling is that it be done so that the sample is accurately 
representative of the whole. One such method, called a convenience sample, might be simply 
using consecutive admissions until a target number is reached, or, if this method presents too 
great a burden for collecting data, using every second, third, or fourth admission is an option. 
Minnesota is a State that uses this approach. Another option is to retrospectively select a random 
sample from an admission or discharge list. Tennessee and Colorado employ this approach. 
Whatever method is used, it is critical that it be applied consistently. Colorado's OMS does a 
random selection from its discharge list. It is important to guard against any method that would 
allow programs to select patients for a sample, since this selection could lead to a biased sample 
that might emphasize a program's achievements and minimize its problems. 


The need for sampling is a pragmatic reality, and sampling will probably be used in most outcomes 















monitoring systems. 


It is also important that the sample not be limited to those patients who complete treatment. 
Consent for participation should be sought shortly after admission so that even patients who drop 
out of treatment early can be interviewed later. A sample solely of treatment completers will not 
be representative of the totality of patients who enter treatment. Obtaining information from 
patients who do not complete treatment is essential to program and systemic improvement. 

Stratified sampling involves an attempt to assure sufficiently large samples of subgroups for data 
analysis. A random method may not produce a sufficient number of persons of color or cocaine 
abusers, for example, in areas where these populations are relatively small. Similarly, a large 
enough sample of pregnant women may not be generated by a random selection of general 
admissions. Depending on the questions to be addressed by the OMS, sampling methods may 
have to be adapted to ensure that questions of great interest can be answered. 


A sample made up solely of treatment completers will not be representative of the totality of patients 
who enter treatment. Obtaining information from patients who do not complete treatment is 
essential to program and systemic improvement. 


Instrument Design and Selection 

The specific kinds of information recommended for an OMS are described in Chapter 5. A State 
has four broad options for meeting its identified information needs: 1) design its own set of 
instruments; 2) use existing instruments; 3) modify existing instruments; or 4) use some 
combination of new, existing, and modified instruments. 

The advantage of new instruments is that they can be tailored to the unique needs of each State's 
OMS. The disadvantages are the costs and time associated with instrument development and 
validation. 

Existing instruments have many advantages, provided they have established validity and 
reliability for the populations being studied. If the instruments are accepted within the research 
community, findings will be less open to challenge. All the expenses associated with instrument 
development have been paid for by other sources. To the extent that the same instruments are in 
use elsewhere, valid comparisons can be made across States and studies. Many widely used 
instruments are in the public domain. (The content for any instrument developed with public 
monies is by law in the public domain; however, there may be costs associated with reproduction 
of forms or software-licensed administration packages.) The content of instruments in the public 
domain can also be modified without permission. The use of copyrighted instruments is typically 
much more expensive, and in some cases costs may be prohibitive for the number required for a 
statewide OMS. On the other hand, the expense of copyrighted instruments may be justifiable if 















the instrument delivers exactly what the planners want and there is no comparable instrument in 
the public domain. 

Modifying instruments offers some of the benefits of designing new ones and some of the 
benefits of using existing ones. Yet, modifications must be made with extreme caution. The 
validity and reliability associated with existing content cannot be assumed to transfer to a 
modified instrument. However, in some cases, sections of an instrument can be deleted with no 
hann done; a few new items could be pilot tested and added. Instrument modifications should be 
done only by experienced instrument developers. 

Many States may look to a combination of existing and new instruments because it is unlikely 
everything they want has already been developed. Whatever package is assembled, a field test 
should be undertaken to try out the instruments and procedures on a small number of cases and 
programs to detennine whether the package will be satisfactory for the State's purposes. 

A few questions will be useful in guiding the evaluation and comparison of existing instruments. 

• What is the overall purpose of the instrument? Is it a patient assessment tool? Does it contain 
an adequate history of AOD use? Does it include other important domains (physical and 
psychological health, criminality, family and social relationships, vocational functioning)? What 
is missing that should be covered? 

• What time interval is addressed? Can the instrument be used at admission and at posttreatment 
followup? 

• Is the instrument designed to be self-administered? Is the reading level appropriate for the 
patient population? How long will it take the average patient to complete? Are the instructions 
to the patient clear, and is the layout designed for ease of completion? 

• Is the instrument a structured interview? How long will the average interview take? How much 
staff training is required? 

• Does the instrument measure treatment services? Are the categories and questions appropriate 
to local needs? Do the response choices allow for adequate characterization of similarities and 
differences? Is the instrument a one-time summary or week-by-week record? Who completes 
the instrument—the patient or staff? 

• Is the instrument in the public domain or copyrighted? How much does it cost? 

• Where else has this instrument been used? Are those users satisfied with the results they got? 
Are there any modifications they would recommend? 

Without knowing the purpose of a specific OMS, it is risky to recommend specific instruments. 
The next chapter suggests some legitimate starting points, however. Different instruments are 
better for different purposes and needs. The challenge is to weigh all the needs identified by 
OMS planners and to determine how best to meet those needs by capitalizing on others' 
experience and instrument development whenever possible. Some States are ahead of others in 
this process, and their examples can provide guidance by illustrating both pitfalls and successes. 

Field Testing 

Treatment providers must be involved in the OMS at the design stage since treatment staff will 
bear the burden of most of the in-treatment data collection. Flaws that appear during the field test 



should be addressed immediately and the appropriate modifications made. The field-testing 
phase requires honest acceptance of whatever flaws may appear and a commitment to respond as 
necessary and not remain wedded to the original design. 

The field test need only involve a small number of programs, but they should be selected so that 
their diversity reflects the range of programs that will be involved in the OMS as a whole. 

Staff Training and Implementation 

Successful implementation of the OMS will involve training and supervision of personnel who 
will collect the information, and the facilitation of conditions under which accurate information 
can be collected. 

Studies that have been done with AOD patients show that accurate information can be obtained 
both during treatment and in followup interviews. Self-report information from AOD patients 
tends to be more accurate under the following circumstances: 1) when the measures are 
standardized, 2) when the data gatherers have been trained to use the instruments consistently, 3) 
when patients are drug free and are not undergoing withdrawal or emotional distress, and 4) 
when patients are motivated to cooperate with the information gatherer (Litten and Allen, 1992). 


Self-report information from AOD patients tends to be more accurate under the following 
circumstances: 

• When the measures are standardized 

• When the data gatherers have been trained to use the instruments consistently 

• When patients are drug free and are not undergoing withdrawal or emotional distress, and 

• When patients are motivated to cooperate with the information gatherer. 


Training is an important part of ensuring that those who will be involved with the OMS are 
invested in its success, and personnel should take the training very seriously. Because of staff 
turnover, training should be available for replacement staff as well those on board when the 
project is initiated. 

Training needs will vary from system to system and will depend upon the involvement of outside 
contractors and the expertise level of State personnel. 

For outcomes monitoring purposes, providers should be trained to collect patient baseline and 
treatment service data in accord with rigorous procedures. Each provider agency should have at 
least one representative who receives direct training in the implementation of the OMS. 

Three phases of training are necessary: 







• Training to orient providers to the overall implementation of the OMS. This basic orientation 
should be directed not only toward data collectors but toward a broad audience of providers 
and single State agency personnel. 

• Technical training for data collectors, including instruction in using the instruments that will be 
part of the outcomes monitoring process. A procedures manual and instrument completion 
manual are essential. 

• Followup or refresher training for training new personnel and assuring continuous quality 
assurance. 

Training can be provided by qualified single State agency personnel, by an independent 
contractor, or through some other available form of technical assistance. While trained provider 
personnel can, to some extent, pass on what they have learned to colleagues who will also be 
involved in implementing the OMS, this "train-the-trainer" model can result in dilution of rigor 
and consequent reduced effectiveness. This reduction can be a real concern in the context of data 
collection and research, since the communication of mistakes or misinterpretations and the 
incorporation of these errors into the use of the OMS could substantially corrode the research 
effort. As much as possible, direct training should be available for all participating providers. 

Training is more feasible if all programs do not start data collection at the same time. A training 
schedule can be set up to stagger individual program participation in the data collection process. 



Chapter 5—Content of an Outcomes 
Monitoring System 

Methods of data collection for an outcomes monitoring system (OMS) have been discussed in 
the previous chapter. Content of an OMS is discussed in this chapter. While these two aspects of 
OMS design are truly inseparable, there are some advantages in considering the "what" 
separately from the "how." Overlap will be evident, however, when content depends on the 
duration of time investigated, and when accuracy of content depends on the timing of data 
collection. These factors, while mentioned in this chapter, were discussed in detail in the 
preceding chapter. 

Determining the content of an outcomes monitoring system depends upon a number of factors: 

• Purpose of the system 

• Specific questions for which answers are sought 

• Types of programs involved 

• Available resources. 

While there is no single, definitive set of variables to include, an OMS needs to describe both 
patients and the treatment they receive. To date, most outcomes studies have placed more 
emphasis on differences in patients than differences in treatment; many have excluded treatment 
variables altogether. In recent years, a major shift has occurred and several well-designed studies 
have been launched to examine the relationship of aspects of treatment to patient outcomes (Ball 
and Ross, 1991; McLellan et al., 1993; Moos et al., 1990). 

The number of patient variables that can be included in outcomes studies is virtually unlimited. 
Because each additional variable translates into additional costs, a basis for making decisions 
must be established. Three questions can be asked with respect to every patient variable under 
consideration for inclusion in the OMS: 

• Can this variable be applied to some interval of time before and after treatment to measure 
change in patient status? In other words, can it be used as an outcome measure? 

• Is this variable a known (or hypothesized) predictor of outcomes? 

• Will the findings related to this variable have any practical applications? 

The first question is important because changes in patient status are the essential components of 
an OMS. The second question is important because variables that are known to predict outcomes 
are crucial to characterizing and distinguishing treatment populations. Predictor variables 
include, for example, the chronicity of substance use, social stability, and criminality. Predictor 
variables must be taken into account in interpreting the outcomes for any particular group of 
patients. In research terms, predictor variables are called independent variables, and outcome 
variables are called dependent variables. 



In research terms, predictor variables are called independent variables, and outcome variables are 
called dependent variables. 


The third question relates to utility. Even when the answer to one of the first two questions is 
yes, the variable may be of limited utility. For example, suppose it had been established that 
people of one particular religious background had better outcomes than another. While religion 
could then be considered a predictor variable, it may have no practical implications for patient 
assessment, treatment placement, or treatment design. The same might be said of a finding that 
patients with life-threatening health conditions had better outcomes. This hypothetical finding 
would be interesting and not necessarily unexpected, but it would have little practical 
application. These examples also illustrate the need for caution when interpreting correlations. 
Correlates are not necessarily causally linked, although they may be. (More research would be 
necessary to make such a determination.) Even when the relationship may be cause and effect, 
utility can still be limited. In the two situations illustrated, for example, no one would suggest, 
based on the findings, that patients be encouraged to change religions or develop life-threatening 
conditions. 

With respect to outcome measures, Chapters 1 and 2 discussed the critical importance of 
including outcome measures that have political and pragmatic utility, such as those that 
document that treatment produces cost savings in areas such as healthcare utilization and the 
criminal justice system. 

The questions related to predictive value and utility can also be applied to proposed treatment 
variables. Is this variable to be included because it is known or believed to be related to patient 
outcomes? If it is related, what might be the implications of findings for patient placement, 
program design, or State policy? 

Before discussing variable selection in more detail, it will be helpful to present and explain the 
way variables are categorized in this chapter. As shown in Exhibit 5-1, patient variables are 
categorized as administrative, predictor, and baseline/outcome. Treatment variables are classified 
as administrative and predictor. This classification scheme is primarily for ease of presentation; 
the distinctions are not necessarily so clearly cut. For instance, some variables classified here as 
administrative or baseline can also be considered predictor variables. 

The two collections of patient variables currently in most widespread use are the Federal Client 
Minimum Data Set, also referred to simply as Client Data Set (CDS), and the Addiction Severity 
Index (ASI). They are explained in detail below. 







Federal Client Data 


The Alcohol and Drug Abuse Client Minimum Data Set was developed by the National Institute 
on Alcohol Abuse and Alcoholism (NIAAA) and the National Institute on Drug Abuse (NIDA) 
to standardize States' reporting of patient data for Federal reporting purposes. This data set is 
required for all treatment providers that receive any State alcohol and/or drug agency funding 
(including Federal block grants), and must be collected on all patients served by these programs, 
regardless of their individual payment sources. The CDS and related data reporting are now 
under the auspices of the Substance Abuse and Mental Health Services Administration Office of 
Applied Studies. 

As shown in Exhibit 5-2, the Client Data Set includes patient and program identifiers, 
demographics, information about AOD use, and other patient and program information. Because 
its use is mandatory for many providers, its development was collaborative, and the variables and 
response choices have already been standardized. The CDS will be presented here for 
consideration as a legitimate starting point for OMS content. It makes no sense to develop a 
competing set of variables and then require treatment providers to use different sets for different 
purposes. However, suggestions will be offered about how to expand, modify, or supplement the 
CDS to meet local needs without violating the integrity and consistency of data reported to 
Federal agencies. 


Addiction Severity Index 

The Addiction Severity Index assesses seven areas of patient functioning: alcohol use, drug use, 
medical status, psychiatric status, employment and financial support status, legal status, and 
family and social relationships. Originally developed in 1980, it was revised most recently in 
1992 (McLellan et al., 1980; McLellan et al., 1992b). Each of the dimensions covered by the ASI 
includes lifetime measures, which can serve as predictor variables, and past-3 0-days measures, 
which can serve as baseline/outcome measures. The ASI also includes clinical and patient- 
reported ratings of problem severity in each of these areas, as well as the patient's rating of the 
need for more treatment or assistance in each area (The fifth edition of the ASI has been 
reprinted in full in a previous Treatment Improvement Protocol (TIP) Screening and Assessment 
for Alcohol and Drug Abuse Among Adults in the Criminal Justice System) 

The validity and reliability of the ASI have been well established for a variety of diverse 
populations (Longabaugh, 1991). Because the ASI provides a standardized way to measure 
problem severity across many dimensions, treatment outcomes across programs and in varied 
geographical areas can be systematically compared. 

According to NIDA, which has supported its development, the ASI has achieved widespread 
acceptance around the world among AOD treatment personnel and researchers. It is in use in 
more than 1,000 programs in the United States and is increasingly being mandated by State and 
local governments to assess government-funded programs (Mathias, 1994). The ASI has the 
additional advantage of having been translated into nine languages. 


Exhibit 5-1 

Patient and Treatment Variables 

Patient Variables 

Administrative 

Predictors 

Baseline/Outcome 

Patient identifier 

Demographics 

AOD use frequency 

Referral sources 

Education 

AOD use amount 

Payment source 

Vocational history 

Mode of drug administration 


Social history 

HIV risk behaviors 


Alcohol and other drug (AOD) 
use history 

AOD treatment history 

Medical history 

Psychiatric history 

Legal problems 

Motivation 

Treatment readiness 

AOD dependence symptoms 

Physical health 

Psychological health 

Employment 

Financial stability 

Legal problems 

Family/social relationships 

Treatment Variables 

Administrative 

Predictors 


Program identifier 

Setting/level of care 




























Admission date 

Therapeutic modality 


Discharge date 

Context 


Length of stay 

Treatment components 


Charges 

Staffing 

Discharge status 

Patient satisfaction 

Posttreatment services 



No single patient assessment instrument will ever achieve universal acceptance; the AOD field is 
too diverse for that. The ASI, like any instrument, has its limitations and its critics. Some of the 
limitations and ways to compensate for them are discussed in the sections that follow. While 
consensus panel members were reluctant to recommend any one instrument as the standard for 
State outcomes monitoring systems, it was proposed that the ASI or its constituent variables be 
given serious consideration because of the strong base of research supporting its validity, 
reliability, and utility. Furthennore, its content addresses most of the areas recommended for 
consideration. 


Patient Variables 

As explained above, patient variables can be historical one-time measures (predictor variables), 
or they can measure status prior to treatment at different points along the treatment continuum or 
following treatment (baseline/outcome measures). For purposes of an OMS, data collected at 
intake or shortly thereafter should include both historical and baseline measures. 


While consensus panel members were reluctant to recommend any one instrument as the standard 
for State outcomes monitoring systems, it was proposed that the ASI or its constituent variables be 
given serious consideration because of the strong base of research supporting its validity, reliability, 
and utility. Furthermore, its content addresses most of the areas recommended for consideration. 


Patient predictor variables include characteristics generally referred to as demographics such as 
gender, age, and race/ethnicity, as well as social/vocational factors such as marital status, 



















employment status, level of education, and income level. Predictor variables indicate special 
treatment needs. These variables also include aspects of a patient’s history that may include such 
factors as a history of physical or sexual abuse, childhood conduct disorder or antisocial 
behavior, age of onset of substance use, number of previous treatments and admissions for 
detoxification, psychiatric hospitalizations, chronic medical conditions, and many other factors. 
These will be discussed in detail later in this chapter. 


Exhibit 5-2 

Federal Client Minimum Data Set (1990) 

Required Items 

Optional Items 

PROVIDER IDENTIFIER 

PRIMARY AOD PROBLEM 

MARITAL STATUS 

PATIENT IDENTIFIER 

Alcohol 

Never married 


Cocaine/crack 

Now married 

DATE OF ADMISSION 

Marijuana/hashish 

Separated 


Heroin 

Divorced 

DATE OF BIRTH 

Methadone 

(nonprescription) 

Widowed 

SEX 

Other opiates and 
synthetics 

LIVING ARRANGEMENTS 

Male 

PCP 

Homeless 

Female 

Other hallucinogens 

Dependent living 


Methamphetamines 

Independent living 

RACE 

Other amphetamines 



Other stimulants 

PRIMARY SOURCE OF 

Alaska Native 

Benzodiazepines 

INCOME/SUPPORT 

American Indian 

Other tranquilizers 


Asian or Pacific 

Barbiturates 

Wages/salary 

Islander 

Other sedatives or 

Public assistance 

Black 

hypnotics 

Retirement/pension 

White 

Inhalants 

Disability 

Other 

Over-the-counter 

drugs 

Other 

ETHNICITY 

Other 

None 

HEALTH INSURANCE 

Puerto Rican 

SECONDARY AOD PROBLEM 


Mexican 

(choices same as 

None 

Cuban 

Other Hispanic 

Medicare 















Not of Hispanic Origin 

HIGHEST SCHOOL GRADE 
COMPLETED 

EMPLOYMENT STATUS 

Employed full time 
Employed part time 
Unemployed 
Not in the labor force 

(Optional 

breakdowns) 

Homemaker 

Student 

Retired 

Disabled 

Inmate of 

institution 

PRINCIPAL REFERRAL 
SOURCE 

An individual (or self) 

AOD abuse care 

provider 

Other health care 

provider 

School 

Employer/EAP 
Other community 
referral 

Court/criminal justice 
referral 

(Optional CJ referral 
categories) 

State/Federal 

court 

Other formal 
adjudication 
process 


primary) 

TERTIARY AOD 
PROBLEM 

(choices same as 
primary) 

AGE OF FIRST USE (OR 

ALCOHOL 

INTOXICATION) 

USUAL ROUTE OF 
ADMINISTRATION 

Oral 

Smoking 

Inhalation 

Injection 

Other 

FREQUENCY OF AOD USE 

No use past month 
1-3 times past month 
1-2 times per week 
3-6 times per week 
Daily 

LEVEL OF SERVICE 

Hospital inpatient 

detoxification 

Free-standing 

residential 

detoxification 

Hospital rehabilitation 

(acute care) 

Short-term (<30 days) 
nonacute residential 
Long-term (>30 days) 
nonacute residential 
Intensive outpatient (2+ 
hours per day 3+ 


Medicaid 

Private insurance 

Blue Cross/Blue Shield 

Health maintenance 

organization 

Other 

Unknown 

EXPECTED PRIMARY SOURCE 
OF PAYMENT FOR THIS 
TREATMENT EPISODE 

Self-pay 

Workers compensation 

Medicare 

Medicaid 

Other government 
payments 

Blue Cross/Blue Shield 

Other health insurance 

No charge 

Other 

Unknown 

DSM-lll-R DIAGNOSTIC CODES 
PSYCHIATRIC PROBLEM 

Yes 

No 

PREGNANT AT ADMISSION 

Yes 

No 

VETERAN STATUS 

Yes 

No 

DAYS WAITING TO ENTER 
TREATMENT 











Probation/parole 

Other 

recognized legal 

entity 

DUI/DWI 

Diversionary 

program 

Prison 

PRIOR AOD TREATMENT 
ADMISSIONS 

0 

1 

2 

3 

4 

5 or more 


days per week) 
Nonintensive outpatient 
Outpatient 
detoxification 

METHADONE PLANNED AS PART 
OF TREATMENT 

Yes 

No 

INITIAL ADMISSION IN 
TREATMENT EPISODE VERSUS 
TRANSFER IN SERVICE 
PATIENT VERSUS CODEPENDENT 
OR COLLATERAL RECORD 


The distinction between predictor variables and baseline/outcome measures is more an issue of 
form than substance. Some variables can fit both categories, depending on when they are 
measured and whether the question is repeated at treatment followup. For example, the number 
of lifetime detoxification admissions can be collected at intake simply as an indicator of chronic 
intoxication. But admissions to detoxification could also be used as a baseline/outcome measure 
if compared over a defined interval before and after treatment (for example, the number of 
detoxification admissions during the 6 months before treatment compared with the number 
during the 6 months after treatment). Similarly, a psychiatric diagnostic variable could be used as 
a predictor variable, while measures of recent psychological distress could be used as 
baseline/outcome measures. 

Administrative Patient Variables 

Patient Identifier 

A patient identifier is critical to linking various forms for the same patient and for linking 
followup questionnaires to treatment questionnaires. It can also be used to match the records of 
multiple admissions for the same patient. 

There are basically two types of patient identifiers—those that can be traced back to an 
individual and those that cannot. Examples of identifiers that can be traced to an individual 
include name, social security number, driver's license number, and permanent numbers assigned 
to recipients of public assistance. The concern about using traceable identifiers in an OMS is that 











their use is a breach of patient confidentiality. Identifiers that cannot be traced to an individual 
include cryptograms comprised typically of some combination of the following: letters from the 
first, middle, and/or last name; date of birth; digits from the social security number; or other 
identifying elements that do not change over time. The drawback of nontraceable identifiers is 
that they cannot be used to match records with other information systems that do not use the 
same type of identifiers (such as Medicaid, child protection, arrest records, and the like). 
Whichever type of patient identifier is used, extreme caution must be taken to protect patient 
confidentiality. All patient identifiers can be "scrambled" upon data entry so they are 
decipherable only to a few designated individuals who know the code. Computer databases can 
also be protected by several layers of security. The legal issues related to protecting 
confidentiality are discussed in Chapter 6. 


Whichever type of patient identifier is used, extreme caution must be taken to protect patient 
confidentiality. 


Referral Sources 

The Federal Client Data Set includes a required item for principal referral source. One alternative 
would be to expand the response choices (for instance, separating self and family and adding 
more specific choices under community and healthcare referrals) and then collapse them for 
Federal reporting purposes. Another alternative would be to have the option "check all that 
apply" in addition to a "principal" referral. The intended use of the data should guide the final 
decision. 

Payment Source 

An optional item in the Federal Client Data Set lists response categories for "Expected primary 
source of payment for this treatment episode." In most States, it would probably be more useful 
to identify the actual payment source or sources. Since this item may not be determined at intake, 
it may more practical to include this question on a discharge form. 

Patient Predictor Variables 

Demographics 

Demographic information such as age, gender, and race/ethnicity has obvious descriptive 
importance in an outcomes monitoring system. To the extent that differences in outcomes might 
be observed among major demographic groups, this information must be taken into account in 
planning services. 

These variables serve a number of useful functions. Aggregated across programs at a State level, 
they provide a descriptive profile of who is entering treatment. They can provide comprehensive 
profiles of patients being served by the treatment system or any portion of it. By comparing this 







in formation with population base rates and AOD incidence/prevalence rates, it is also possible to 
use these data to determine who is not being served by AOD treatment programs. 

The patient demographic profile can also be used to track trends in treatment admissions over 
time. Changing patterns might suggest the need for more programs designed to serve special 
populations. For example, if an increasing number of women are entering treatment, this increase 
may indicate a need for more programs to serve this population. Similarly, if a decreasing 
number of adolescents are receiving AOD treatment, the capacity for this group may need to be 
decreased. 


The patient demographic profile can also be used to track trends in treatment admissions over time. 
Changing patterns might suggest the need for more programs designed to serve special populations. 


Demographic information can also be cross-tabulated or correlated with other variables such as 
AOD use patterns and treatment discharge status to detennine whether demographic factors are 
independent predictors of outcomes or interact with other variables to predict outcomes. 
Controlling for demographic factors in data analyses can also indicate when demographic 
predictor variables are merely artifacts of other predictor variables. 

Frequently in social science research, a demographic factor is significantly associated with an 
outcome. For example, race may be significantly associated with likelihood of relapse. However, 
race may also be significantly associated with a primary drug problem. For instance, the rate of 
treatment admissions for cocaine addiction may be higher among blacks than whites in the 
geographic area under study. An analysis that controlled for drug of choice might find that once 
drug of choice was controlled for, there was no significant difference between the outcomes of 
blacks and whites; that is, whites and blacks with alcohol problems had comparable outcomes, 
and whites and blacks with cocaine problems had comparable outcomes. 

Typically, the kinds of variables included under demographics are the easiest to turn into specific 
questions on data collection instruments. They are usually formatted as categorical variables, 
where one response is selected from among a list of responses, and thus are much easier to 
design than measures of problem severity. Nonetheless, some important considerations should 
not be overlooked. These will be explained in the following discussion when pertinent. 

Gender and age. Gender is an essential variable and one of the few that needs no discussion. 
Age is also extremely important as a predictor of outcomes and relatively straightforward to 
collect. In some systems that include birth date and date of admission, age is computed rather 
than recorded. 

Race/ethnicity. Many instruments include the race breakdown used in the 1990 U.S. Census and 
the Federal Client Data Set. Five racial categories are used (Alaska Native, American Indian, 
Asian or Pacific Islander, Black, and White) with an extra response choice for "Other"; a 
separate variable addresses Hispanic ethnicity. Hispanic people can be of any race. While these 







formats have advantages in terms of standardization, they can pose some problems. Biracial and 
multiracial patients are sometimes categorized into one race by the interviewer based on 
appearance; sometimes patients are asked to pick which racial category best describes them; and 
sometimes biracial and multiracial patients are categorized as "other." 

An alternative to the forced choice option is a "Check all that apply" response format. Several 
choices can then be formatted into single categories for Federal reporting purposes, but the value 
of more accurate and detailed descriptions would not be lost. 

Another potential problem with existing standard racial/ethnic categories is that they may not 
provide sufficient information for some localities. For example, agencies in States with relatively 
large American Indian populations may wish to include tribal affiliation as a subset of the Indian 
category. States with diverse Asian American populations may find it helpful to distinguish 
Southeast Asian and other Asian subgroups. For example, forms used in California and 
Washington State include more than 10 specific Asian designations. 

On the other hand, some persons argue against collecting data on race because results can be 
misinterpreted or used for political ends. Findings attributed to race actually often reflect cultural 
or socioeconomic factors. It is important to remember that even widely used variables and 
response choices should be examined in terms of whether they will provide information in ways 
most appropriate for local purposes. 

Primary language/immigration status. Language difference is another variable that may help 
State planners design programs that are appropriate to the populations they serve. The 
Washington State system includes a question asking whether use of English is functional or 
limited, as well as an item identifying the primary functional language from among a list of 46 
languages, including American Sign Language. Recent immigration status may also be important 
for cultural considerations and special needs, particularly in those States with large numbers of 
immigrants. 


Some persons argue against collecting data on race because results can be misinterpreted or used for 
political ends. Findings attributed to race actually often reflect cultural or socioeconomic factors. 


Education 

The Federal Client Data Set uses the highest school grade completed as a measure of education 
level (a general equivalency diploma is equivalent to 12 years). Another option is a categorical 
variable that includes as response choices the highest degree attained: none, high school diploma, 
vocational technical certificate, bachelor's degree, master's degree, or professional degree (e.g., 
M.D., J.D., or other doctoral degrees). 







Social History 


Marital/relationship status, parenthood, sexual orientation. Marital status is an optional item 
on the Federal Client Data Set. The item as it exists does not provide a category for persons 
living in a committed relationship but not legally married. OMS planners might want to consider 
different response choices for this existing item or perhaps add an additional item. 

Information on parenthood status and number of dependents can also help with program design 
and may be predictive of outcomes. The Washington State forms include questions about the 
number of children living with the patient (their own and others'), as well as children not living 
with the patient. 

Sexual orientation can be a controversial item, and there may be doubt about the truthfulness of 
patients' responses because of their fears of discrimination. Some proponents argue that it is 
important to determine whether gay, lesbian, bisexual, or transgender individuals are more likely 
than others to have AOD problems. It is important to remember, of course, that treatment 
admissions are not an accurate reflection of AOD abuse prevalence. If underreporting were to 
occur, it would distort the findings anyway. As with all other variables under consideration, the 
key question is utility. Is the question asked routinely as part of individual treatment planning? 
Would the system be changed in any way as a result of findings in this area? If the data are not 
going to be put to practical use, it is probably not justifiable to ask patients to reveal such 
sensitive information. 

Living arrangement. One simple breakdown is provided as an optional item in the Federal 
Client Data Set: homeless or transient, dependent living (as with parents or in a supervised 
setting), or independent living. These latter two response choices could be broken down into 
subcategories. For instance, dependent living could include separate response choices for living 
with parents, living with other relatives, board and lodging, halfway house, and so forth. 
Independent living could include separate choices for house, apartment, or mobile home. The 
utility of more detailed information should guide the decision process. 

Veteran status. Veteran status is also an optional item in the Federal Client Data Set. The item 
is easy to collect and noncontroversial but may have limited utility overall. Minnesota modified 
the item so the "Yes" response was changed to "Yes, no combat" and "Yes, served in combat 
zone" to see whether combat experience might be a predictor variable. Iowa has modified this 
question to address military status, with response choices of "none," "veteran," "in reserves," and 
"active duty." 

Socioeconomic status, income, health insurance. Socioeconomic status (SES) is a useful 
concept but not necessarily an easy one to operationalize. OMS planners may want to review 
available SES classifications to detennine whether any would be useful for their purposes. 
Several optional items from the Federal Client Data Set can be used as rough SES indicators, for 
example, primary source of income/support, health insurance, and expected source of payment 
for treatment. Other options include personal or family income and occupational level. The State 
of Washington asks for monthly household income and monthly personal income. Iowa asks for 
taxable individual monthly income as well as occupation level (professional/managerial, 


sales/clerical, crafts/operatives, nonfarm laborers, farm owners and laborers, and 
service/household). 

Health insurance variables can be useful since health insurance plays a part in dictating the kind 
of healthcare to which a person has access, which may have a significant relationship to 
outcomes. While any one variable in this general area has its limitations, a selection of several of 
these variables will probably provide sufficient information for categorizing patients. 

Spirituality/religious affiliation. Another factor that can be considered as part of social history 
is religious affiliation and spirituality. Recovering AOD abusers often emphasize the importance 
of spirituality in their recovery. Involvement in church activities is another predictor of positive 
outcomes. Many AOD treatment programs use the 12-step approach, and issues of religiosity and 
spirituality may be important and can be measured (Tonigan et al., 1991). 

Family and Other Social Relationships 

The ASI addresses several aspects of family and social relationships. As lifetime measures, it 
determines whether close personal relationships existed with parents, siblings, partner(s), 
children, and friends. It also asks about periods of serious problems with these people as well as 
with other family members, neighbors, and coworkers. (This section, like all others in the ASI, 
also includes many 30-day measures for baseline/outcome comparison.) 

Other variables could certainly be used. The key question, in terms of both treatment planning 
and predictive value, would be the availability and extent of social supports. 

History of physical/sexual abuse. Many adolescents and adults in treatment report histories of 
sexual abuse and family violence. It is not yet known to what extent these factors may be 
predictors of outcome. Documentation of the extent of these problems among AOD patients, 
however, can be used for treatment planning and design purposes. 

Many instruments, including the ASI, require a yes or no response to questions about abuse, 
victimization, and/or perpetration. For example, the Washington questionnaire asks whether the 
patient has ever been a victim of domestic violence and whether the patient is currently a victim 
of domestic violence. While this format allows for quantification of the extent of these problems, 
it may be inadequate as a predictor of treatment outcome. Many factors contribute to the impact 
of physical and sexual abuse on its victims: identity of the perpetrator, age of the victim, 
frequency and severity of abuse, and whether the victim sought or received help at the time or 
since the abuse. Assessment of all these issues, while appropriate for clinical purposes, would be 
beyond the scope of an OMS. 

Abuse victimization and perpetration can also be used as baseline/outcome measures and will be 
discussed in this context later in this chapter. 


Vocational History 


Employment status. A variable that can present problems is employment status. The Federal 
Client Data Set includes a forced-choice response format: employed full time, employed part 
time, unemployed (and looking for work), or not in the labor force. This last category can be 
expanded into another forced-choice format that includes homemaker, student, disabled, retired, 
and correctional inmate. A forced choice results in the loss of information for patients who have 
multiple roles and responsibilities, such as the homemaker who is employed and goes to school, 
or the student who is also employed. Multiple role status in itself may be a predictor of outcomes 
and thus valuable information to retain. Forced choices by their very nature reflect a value as to 
which is more important when multiple responses apply, a value which may not be shared by 
patients or other OMS stakeholders. In the case of the Federal Client Data Set, "employed full 
time" would supersede choices listed below, and other applicable information would be lost. 

Loss of information can be avoided with a "check all that apply" format. For Federal reporting 
purposes, the responses can be recoded into the hierarchy of response choices set forth in the 
Client Data Set guidelines. While this TIP cannot address the nuances of all variable options in 
this detail, the employment status variable, like the race variable, points out the pitfalls of 
assuming that demographics are easy variables to define or agree on. 

Employment history. Current employment status does not provide information about 
occupational stability over time or give any indication of downward mobility that might be 
associated with AOD use. The ASI includes variables on length of longest held full-time job, 
most recent or current occupational status, and usual employment pattern over the previous 3 
years. The Iowa system inquires about the number of months worked during the previous 6 
months. These are examples of variables that inquire beyond employment status at admission. 


Current employment status does not provide information about occupational stability over time or 
give any indication of downward mobility that might be associated with AOD use. 


AOD Use History 

Information about the history of substance use is important for proper evaluation of treatment 
outcomes because the type of drug and the severity of the AOD problem may dictate the 
intensity and specificity of treatment as well as predict outcome. People with a substance use 
disorder are a heterogeneous population with a variety of different use patterns; however, the 
effects of all these differences on outcomes are unclear. 

Because reduction in substance use and its associated consequences is one of the primary goals 
of treatment, many substance use variables are appropriate as baseline/outcome measures and 
will be discussed later in this chapter. However, some substance use variables can be used as 
predictors as well, and some are solely one-time measures. 







Substances used, primary drugs of abuse. Some indication of the substances used by 
individual patients is essential to an OMS. There are various ways to collect this information, 
and several combine type of drug (or drug class) with use frequency. The Federal Client Data Set 
requires coding the drug (or drugs) identified as primary problem, secondary problem, and 
tertiary problem for each patient. Clinical judgment is used to make the determination. For 
primary and secondary drugs, codes are also required for age of onset of use (first intoxication 
for alcohol), frequency of use, and usual mode of administration. This data set has some 
limitations, which will be discussed below. 

Age of onset. Age of onset of use has been shown to be a predictor of substance use severity in 
both epidemiological and clinical studies (Buydens-Branchey et ah, 1989). In the Federal Client 
Data Set, this variable is required only for primary and secondary drugs of abuse. If the patient 
initiated use with a third drug, the required information does not provide actual age of onset of 
any AOD use. This limitation can be resolved by either asking for this directly or including age 
of onset for all drug or drug categories reported by the patient. 

Frequency of use. In the Federal Client Data Set, this variable is also required only for primary 
and secondary drugs of abuse. If the patient uses a variety of drugs, no information is obtained 
on the remainder. The variable is further limited by focusing only on recent use. A solution to 
both these problems is to add frequency of use as a variable for all drugs reported. It may be 
preferable, however, to exercise this option as a baseline/outcome measure so that frequency of 
use over a specified time interval can be compared before and after treatment. Use frequency will 
be discussed in greater detail in the section on baseline/outcome measures. 

Mode of drug administration. "Usual mode of administration" is included in the Federal Client 
Data Set. Because it is required only for primary and secondary drugs of abuse, it does not serve 
to identify all current (or previous) injection drug users. It only identifies those clients whose 
usual mode of administration is injection. This information may be deemed inadequate, 
considering the risks for HIV infection associated with even infrequent use, or previous use 
dating back many years. Several States, including California, Iowa, Minnesota, and Washington, 
have included a more general question to address any history of injection drug use. This question 
need not be drug specific. 


Age of onset of use has been shown to be a predictor of substance use severity in both 
epidemiological and clinical studies. 


AOD diagnoses. Substance use disorder diagnoses recorded at intake or discharge may be useful 
for purposes of population profiles. The Diagnostic and Statistical Manual of Mental Disorders, 
Third Edition, Revised (DSM-III-R) (American Psychiatric Association, 1987) codes are optional 
variables in the Federal Client Data Set. However, diagnostic categories alone are of limited 
value as predictor variables and should be supplemented by other indicators of problem severity 
such as frequency of use and use consequences. The full series of AOD abuse/dependence 







diagnostic codes should be listed with instructions to "check all that apply" in order to capture 
the full range of applicable diagnoses for poly drug-using patients. 

Treatment History 

Treatment history is an important patient variable to measure. It can provide an efficient 
indicator of the severity of a patient's alcohol or drug problem. The number of lifetime treatment 
admissions and number of lifetime detoxification admissions, for example, can assist with 
treatment placement decisions and may be useful predictors of outcome. The number of prior 
AOD treatment admissions is included in the Federal Client Data Set. Iowa limits the period of 
inquiry to 10 years. 

Medical History 

In many studies, a relationship has been found between AOD use and injuries, physical 
disabilities, and other medical problems (Cherpitel, 1992; Cregler, 1989; Eckardt et ah, 1981; 
Heinemann, 1993; Moessner, 1979). Whether to collect information on medical history and 
current medical conditions and the amount of detail to include will depend on the purposes of the 
OMS and the availability of resources. Background information could include such data as 
number of lifetime hospitalizations (excluding childbirth) or a general rating scale of overall 
physical health. On the other hand, if more detail is sought, questions could be asked about the 
nature and duration of illnesses or symptom severity. 

While it is true that many illnesses are associated with substance abuse, the purpose of collecting 
exhaustive medical data in an OMS must be justified. What might be appropriate for a medical 
workup is not necessarily appropriate for an outcomes monitoring system. This is another area 
that could generate an almost unlimited number of potential variables, and decisions must be 
guided by the utility of the information in analyzing treatment outcomes. If the purpose is to 
determine whether ill health is related to outcomes, general indicators will probably suffice. If 
the purpose is to determine to what extent medical problems remit with treatment, a 
baseline/outcome measure would be needed. 


The purpose of collecting exhaustive medical data in an OMS must be justified. What might be 
appropriate for a medical workup is not necessarily appropriate for an outcomes monitoring system. 
Decisions must be guided by the utility of the information in analyzing treatment outcomes. 


The ASI includes questions about lifetime medical hospitalizations, chronic medical problems, 
and regular prescription drug use. The Washington system includes questions covering numbers 
of emergency room visits, outpatient/clinic visits, inpatient hospitalizations, and days of 
hospitalization in the previous year. Washington also has separate questions that address whether 
or not the patient is currently under medical care for infectious disease, traumatic injury, 
continuing illness, or dental problems. 







Pregnancy status. Pregnancy status is an optional and controversial item in the Federal Client 
Data Set. In some States, legislation has been passed that mandates reporting certain illicit drug 
use by pregnant women to local authorities. This requirement may motivate women to deny 
pregnancy. Depending on whether this item was completed at intake or shortly thereafter, 
pregnancy may also be underreported. Some women may discover they are pregnant at a later 
point during treatment (or not know at all). 

Psychiatric History 

Psychiatric disorders are found in AOD-abusing populations at rates higher than in general 
populations (Helzer and Pryzbeck, 1988; Hovens et ah, 1994; Regier et ah, 1990; Rounsaville, 
1990; Stowell and Estroff, 1992). Psychiatric disorders may predate AOD abuse, be a 
consequence of AOD abuse, or coexist with AOD abuse. Psychiatric disorders include 
depression and bipolar disorder (manic-depression), anxiety disorders, schizophrenia and other 
thought disorders, posttraumatic stress disorder, dissociative disorders, and eating disorders and 
other compulsive behavior disorders. They also include personality disorders such as antisocial 
personality disorder. (For more information about dual diagnosis of psychiatric disorders and 
AOD abuse, refer to another TIP in this series, Assessment and Treatment Planning for Patients 
With Coexisting Mental Illness and Alcohol and Other Drug Abuse) 

As with physical disorders, it is important to be selective about the amount of information 
collected about psychiatric disorders and emotional distress. Collecting and recording data for an 
OMS is not the same as clinical assessment or documentation. Again, the guiding principle in 
OMS planning is to select those variables that may be significantly associated with treatment 
outcome or that serve as baseline/outcome measures. For example, the amount of subjective 
distress experienced by a patient may have a greater relationship to outcome than to the specific 
nature of the psychiatric diagnosis, or a combination of the diagnostic code and clinical severity 
rating may provide the most useful information. Ultimately, for purposes of data analysis, large 
numbers of variables must be distilled into a smaller number of useful ones. A review of existing 
studies and outcome monitoring systems may guide the distillation process. Incorporating a 
variety of psychiatric assessment instruments into the OMS may seem like a safe way to be sure 
all the bases are covered, but it is also likely to pose an insurmountable obstacle in terms of staff 
and patient time and data analysis. 


The amount of subjective distress experienced by a patient may have a greater relationship to 
outcome than to the specific nature of the psychiatric diagnosis, or a combination of the diagnostic 
code and clinical severity rating may provide the most useful information. 


The Federal Client Data Set includes an optional variable that merely records the existence of a 
psychiatric problem. This question is probably too general to be of any predictive value and too 
vague to provide useful interpretation of findings. Washington includes an item to record 
whether or not a psychological evaluation was conducted and a psychiatric diagnosis was made, 







as well as yes/no questions on the current use of mental health services and psychotropic 
medication. 

The ASI includes questions on the number of lifetime psychiatric hospitalizations and courses of 
outpatient treatment. The ASI also asks about lifetime episodes of depression, anxiety, 
hallucinations, memory or concentration difficulties, difficulty controlling violent behavior, 
suicidal thoughts, and suicide attempts, as well as the use of psychotropic medication. The 
Minnesota OMS added a question to the ASI series to address other compulsive behaviors such 
as eating disorders and gambling. Similarly, Iowa asks a general question about addictions other 
than AOD dependence. 

Motivation to Enter Treatment, Treatment Readiness, and Coercive Influences 

Patients enter treatment for many reasons. Factors that influence the decision may be internal or 
external, and for many patients a variety of factors may impinge on the decision. For some, the 
admission is involuntary; they are court ordered or committed to treatment. For others, coercion 
may be more subtle: the threat of a spouse's or partner's leaving or the loss of a job, a 
professional license, or custody of children are often significant factors. Patients may also enter 
treatment because they are sick, broke, or weary of maintaining their addiction. 

An OMS may include a measure of the coercive influences and motivations to enter treatment in 
order to determine their relationship to treatment outcomes. Findings in this area could have 
important policy implications and could be used to improve patient outcomes. A rating of patient 
readiness for treatment could also be included. If patient readiness were found to be a strong 
predict or of positive outcomes, it might indicate a change in patient placement strategies or the 
need to search for ways to improve patient readiness. 

In Minnesota, the OMS includes a question that addresses the primary condition surrounding 
admission to treatment. Response choices include treatment versus incarceration or as a 
condition of probation or parole, to avoid loss of children or to regain their custody, to prevent 
loss of a relationship or living situation, to regain a driver's license, to keep a professional license 
or job or to stay in school, to retain eligibility for government benefits, or to escape financial 
pressures related to continued alcohol and drug abuse. 

Baseline/Outcome Variables 

The key to valid and meaningful baseline/outcome measures is a comparable interval of 
measurement before and after treatment; Chapter 4 addresses this issue in detail. The advantages 
of short windows, such as 30 days, are more accurate recall and finer precision of measurement. 
A drawback is the lack of information for a longer duration. In fact, this lack is one of the 
criticisms of the ASI, which measures AOD use frequency only for the previous 30 days. These 
questions, asked 6 months after discharge from treatment, would not reveal whether the patient 
had used any alcohol or other drugs since treatment. In contrast, a 6-month interval covers a 
longer duration but lacks precision in terms of measurement. The recommended solution is a 
combination of both short-term and long-term intervals. This combination will be explained 
more in the discussion of specific variables that follows. 


Frequency of AOD Use 


The most obvious baseline and outcome measures are those associated with substance use and 
abuse before and after treatment. Measures of frequency of AOD use are almost always included 
in treatment outcome studies. Frequency of use is important to measure because it is associated 
with a range of biopsycho-social problems. Reduction of use frequency is a useful outcome 
measure even in the absence of total abstinence. A number of instruments incorporate effective 
ways to ask about frequency of AOD use. The ASI includes use frequency for a variety of drugs 
and drug categories but limits the inquiry to the previous 30 days. 


Measures of frequency of AOD use are almost always included in treatment outcome studies. 
Frequency of use is important to measure because it is associated with a range of biopsychosocial 
problems. 


Minnesota's OMS design addressed the limitations of both the Federal Client Data Set and the 
ASI in measuring AOD use frequency. For all the Client Data Set drug categories, use frequency 
responses are: 

• No use in past month 

• 1-3 times in past month 

• 1-2 times per week 

• 3-6 times per week 

• Daily. 

The Minnesota OMS replaces the "no use in past month" response choice with three other 
choices: 

• Used in past 2-6 months (but not in past month) 

• Used in lifetime (but not in past 6 months) 

• Never used. 

This design captures the information as required for the CDS but also allows for a record of 6- 
month and lifetime use. (The lifetime response choice is used only at admission, not at 
followup.) 

AOD Use Amounts 

The amount of drug use per typical occasion of use can be an indicator of the severity of the 
AOD problem and the individual's ability to maintain control over use. Amount of use is not 
used as often as frequency of use because it is more difficult to quantify and typically less valid 
and reliable. For example, while quantity of alcohol consumed during a typical drinking occasion 
may be a useful measure for social drinkers, its value diminishes for persons who abuse or are 
dependent on alcohol. Alcoholics may drink from the bottle, so a measure of number of "drinks" 








becomes meaningless. Even when persons drink cans of beer, glasses of wine, or mixed drinks, 
their accuracy in reporting quantity is likely to decline with increased consumption (Webb et ah, 
1991). 

Amount of use of illicit drugs is even more difficult to quantify. There is no standard measure of 
drug use comparable to that for alcohol. Potency is not controlled by governmental regulation. 
Amount of consumption (number of joints, hits, lines, or fixes) may provide some indication of 
the compulsiveness of the drug-taking behavior, but it does not indicate how much of the drug 
was actually ingested. Drug quantity is not recommended as an OMS measure. 


The amount of drug use per typical occasion of use can be an indicator of the severity of the AOD 
problem and the individual's ability to maintain control over use. 


Mode of Drug Administration 

Mode of drug administration is an important variable for a number of reasons. Injection drug use 
is an indication of problem severity and also important in terms of the risk of transmission of 
HIV and other infectious diseases. The Client Data Set includes "usual mode of administration" 
at admission and the same questions could be asked at followup. Another option is to specifically 
ask for the frequency of injection (the mode of administration of greatest concern) for an 
identical time interval before and after treatment. 

HIV Risk Behaviors 

Injection drug use has already been addressed. In Minnesota, another question addresses the 
frequency of using "clean works" among patients who report injection drug use. High-risk sexual 
behaviors have also been associated with AOD use, either because of poor judgment under the 
influence or the use of sex as a commodity to obtain drugs. Questions can address the use of 
condoms during sexual intercourse and the number of sexual partners over a specified interval 
before and after treatment. 

These are obviously highly sensitive topics. In Minnesota, where questions about using clean 
works are included in the OMS, these questions are the ones most likely to draw objections from 
former patients during followup telephone interviews. Nonetheless, they may prove useful 
measures of behavior change. 

AOD Use Dependence Symptoms 

Measures of symptoms of dependence are useful in detennining the severity of a relapse 
following treatment. Dependence symptoms include: 

• Loss of control over use 

• Unsuccessful attempts to control use 







• Tolerance 

• Withdrawal symptoms. 

According to the fourth edition of DSM (American Psychiatric Association, 1994), a remission 
has been achieved when abuse and dependence symptoms have been absent at least 1 month. 
Early remission covers the first 12 months without symptoms, and sustained remission covers the 
period beyond 12 months. Remission can be further categorized as partial or full. In full 
remission, no abuse or dependence symptoms are present; in partial remission, full criteria for 
dependence are not met, but at least one criterion is present either intermittently or continuously. 
Including DSM-IV criteria symptoms in the OMS will allow for classification of patients' 
posttreatment status in terms of partial or full remission (or partial or full recovery). 

The Federal Client Data Set includes diagnostic codes from the third revised edition of DSM as 
optional items. (In DSM-IV, the latest edition of DSM, the diagnostic codes for abuse of and 
dependence on particular substances are the same as in the third revised edition.) However, 
diagnosis is likely to be established in a clinical setting as a routine part of the assessment 
process. Including all the relevant questions to establish diagnosis at followup may be too time 
consuming, relative to the benefits of this particular element. The Minnesota OMS limits inquiry 
in this area to a question on the number of times the patient experienced withdrawal symptoms in 
a 6-month period before and after treatment. 

Physical Health Measures 

AOD use can have direct and indirect effects on physical health. Direct effects include, for 
example, infections from injection drug use and damage to the liver and other organs from 
alcohol. Indirect effects result from poor nutrition, unsanitary living conditions, lack of sleep, 
and other health risks associated with AOD use and AOD-related lifestyles, such as traumatic 
injury. 

It is sufficient for purposes of an OMS to measure health status before and after treatment, 
without necessarily linking health status to AOD use. For example, the ASI asks, "How many 
days have you experienced medical problems in the past 30 days?" as well as "How troubled or 
bothered have you been by these medical problems in the past 30 days?" and "How important to 
you now is treatment for these medical problems?" Reduction in problem days or subjective 
severity rating can be used to show improved status after treatment. 


It is sufficient for purposes of an OMS to measure health status before and after treatment, without 
necessarily linking health status to AOD use. 


Other options are questions that record the number of days in the hospital or visits for emergency 
care during, for example, a 6-month period before and after treatment. 







General medical care may actually increase for a time during or after treatment as patients attend 
to long-neglected health issues. For this reason, measures of outpatient physician or clinic visits 
do not always show a posttreatment reduction. Inpatient and emergency care, in contrast, 
typically show dramatic decreases (California Department of Alcohol and Drug Programs, 1994; 
Harrison and Hoffmann, 1989). 

Psychological Health Measures 

As with physical health, AOD use can have direct and indirect consequences for psychiatric 
status and emotional well-being. AOD use can exacerbate existing mental health problems or 
bring on depression and difficulties with memory or concentration. Living conditions and 
disrupted interpersonal relationships associated with severe addiction can also cause emotional 
distress. Finally, difficulty controlling violent behavior, suicidal thoughts, and suicide attempts 
can also be associated with AOD use. 

The ASI asks about a variety of psychological symptoms over the previous 30 days, information 
which provides useful baseline/outcome measures. The psychological questions parallel the 
medical questions regarding number of days (in the past 30) on which emotional problems were 
experienced, a subjective rating of problem severity, and a rating of the need for treatment. 

Psychiatric hospitalization over a 6-month period or longer could be used as a baseline outcome 
measure; however, base rates are relatively low, limiting the utility of this item. Outpatient visits 
may actually increase as patients follow through on treatment plans that include attention to 
mental health as well as AOD use. 

Employment Measures 

Current employment status, a Federal Client Data Set item, can be used at admission and at the 
posttreatment contact. Months of employment in the preceding 6 months is another option. 

The ASI uses a 30-day measurement window and several variables: days paid for working, net 
employment income, days experiencing employment problems, subjective rating of employment 
problem severity, and rating of the need for counseling for problems related to employment. 

Financial Stability Measures 

Relying solely on employment variables to assess financial stability is inadequate because a large 
number of patients may not be in the work force (such as students, homemakers, retired, and 
disabled persons). Measures of financial stability can be used to indicate change in status before 
and after treatment. The ASI includes 30-day income measures for unemployment compensation, 
public assistance, and pensions or social security. 

The Minnesota modification of the ASI incorporates questions about financial stability 
comparable to the employment questions: days experiencing financial problems, subjective 
rating of financial problem severity, and rating of the need for assistance with financial 
problems. 


Legal Problem Measures 


Legal problems can be quantified by using measures such as number of arrests over a specified 
interval before and after treatment. Researchers with experience in this area often prefer arrests 
to actual crimes because arrests are a matter of public record, making it more likely that patients 
will give an honest accounting. A recent California study was very successful at quantifying 
actual crimes (California Department of Alcohol and Drug Programs, 1994), but the typical 
OMS is not likely to have comparable resources available. 


Legal problems can be quantified by using measures such as number of arrests over a specified 
interval before and after treatment. 


The ASI uses two 30-day measures: number of days detained or incarcerated and number of days 
engaged in illegal activities for profit. It also includes the patient's subjective rating of severity of 
legal problems and need for counseling or referral for those problems. 

Family/Social Relationships 

Marital or relationship status can be used at baseline and at outcome, but this information can be 
used to indicate whether the status is an improvement. Living arrangement may also be of some 
limited utility. It may be more helpful to measure satisfaction with marital/relationship status and 
living situation. 

The ASI includes 30-day questions about interpersonal conflicts that provide outcome measures, 
but the absence of conflict may mean merely lack of social contact rather than improved 
relationships. The ASI also includes 30-day measures of sexual, physical, and emotional abuse— 
whether or not the patient has been abused or has abused anyone else. 

The Minnesota modification of the ASI adds 30-day measures regarding family stressors other 
than conflict. It also includes a rating of how troubled the patient has been by loneliness or being 
alone, as well as questions about living with people who use alcohol or other drugs or who have 
AOD problems. 

The limitation of these instruments is that neither the ASI nor Minnesota's modification 
addresses changes in positive aspects of family or social relationships, especially the 
establishment of new relationships by patients who had previously been isolated or estranged 
from others. 

Treatment Variables 

Much remains u nkn own about how outcomes relate to the type and quantity of treatment 
received, and this relationship is an issue of concern to the single State agencies. In outcomes 







research, treatment variables have typically been neglected (Sobell et ah, 1987), but they have 
important implications for understanding outcomes. Certainly they have important cost 
ramifications, as in the areas of setting and staffing. If different treatment modalities or programs 
can deliver comparable quality of care as measured by comparable outcomes, then cost alone 
might be the basis for choosing one over another. 

If only baselines and outcomes are considered, then treatment itself is the "black box," with what 
goes on within treatment remaining a mystery to outcomes evaluators and policymakers. For an 
OMS to function effectively, there must be a level of specificity about what goes on between 
patient admission and discharge. In the past, for example, researchers have been preoccupied 
with comparing the results of outpatient versus inpatient programs (Cummings, 1991; Miller and 
Hester, 1986), without looking at the actual content of these programs. Issues that must be 
considered include differences in forms of treatment, such as individual, group, and family 
therapy; differences in settings and populations; differences in lengths of stay and frequency and 
intensity of treatment sessions; and differences in the actual components of treatment. 


If only baselines and outcomes are considered, then treatment itself is the "black box," with what 
goes on within treatment remaining a mystery to outcomes evaluators and policymakers. 


Administrative Treatment Variables 

Program Identifier 

An identifier is necessary to distinguish one treatment provider from another. This is a 
noncontroversial item. 

Admission Date, Discharge Date, and Length Of Stay 

For inpatient and residential treatment programs, length of stay can be computed if the admission 
date is recorded on an intake form and the discharge date on a discharge form. For outpatient 
programs, however, this information only indicates length of time from start to finish of 
treatment and gives no indication of number of contacts during that interval. Hours of 
involvement or contact days could be recorded on a discharge form. Service amounts will be 
discussed in more detail later. 

Level of Service 

Level of service is a required item in the Federal Client Data Set. This breakdown or an 
expansion of it should be included in an OMS. The CDS also requires an indication of whether 
an admission is the initial admission in a treatment episode or a continuation of treatment 
resulting from a transfer (such as inpatient to outpatient care). Level of care can also be 
considered a predictor variable. 







Treatment Charges 


Information on costs associated with treatment is essential to an OMS focused on accountability 
for taxpayer dollars. Information could be collected as a package cost for the whole program or 
as itemized costs for specific services. Costs are not readily available to the program staff who 
complete patient forms, however. As some States move to managed care based on capitated 
rates, costs for individual treatment admissions may be difficult or impossible to compute. 


As some States move to managed care based on capitated rates, costs for individual treatment 
admissions may be difficult or impossible to compute. 


Treatment Predictor Variables 

Therapeutic Modality and Social Climate 

Little work has been done in determining the effect of therapeutic focus on patient outcomes for 
different kinds of patients. For instance, while therapeutic communities differ from 12-step 
programs, there is still not enough in formation available to predict which patients would do best 
in which modality. 

Social climate may also affect patient outcomes. For example, a well- kn own program may be 
extremely similar in structure and focus to many other programs, but its high profile may lead 
patients to have greater expectations of success. 

Treatment Components 

While a measure of therapeutic focus might be important, it might be more revealing to include a 
variety of measures of the specific services that comprise the treatment regimen. Most programs 
actually consist of a composite of some of the following components: 

• Individual, group, or family counseling sessions 

• Medical and psychiatric/psychological evaluation and treatment 

• Educational lectures, films, or reading material 

• Dietary and nutritional guidance 

• Education, referral, counseling, testing, and treatment for HIV/AIDS, tuberculosis, and other 
infectious diseases 

• Educational and vocational services 

• Social services including housing, childcare, and transportation 

• Legal assistance 

• Recreational and occupational therapy 

• Independent skills training 

• Biofeedback 

• Acupuncture 








• Psychotropic medication 

• Opioid substitution therapy 

• Exercise 

• Spiritual counseling 

• Others. 

Some treatment components may be specific to cultural heritage and traditional healing 
practices. 

Decisions must be made not only about which treatment components should be measured, but 
also how to measure them. While instruments that attempt to define and quantify treatment 
components are relative newcomers in the field compared with those that assess patient 
characteristics, they should be examined for their usefulness in this area. For instance, a simple 
checklist could be used to indicate whether or not a patient had received a service, while a more 
sophisticated measure could be developed to describe the frequency, intensity, and duration of 
the service. It would be important to know, for example, whether or not a certain minimum 
amount of a service were necessary to achieve any beneficial effect, or conversely, whether or 
not a threshold were reached beyond which there were little or no measurable benefit. While no 
OMS could measure all the potential treatment variables that could be generated, attention to 
selected key variables may produce useful in formation for allocating limited resources, placing 
patients in the most appropriate settings, and giving them the services most likely to produce 
favorable outcomes. 


It would be important to know whether a certain minimum amount of a treatment service were 
necessary to achieve any beneficial effect, or conversely, whether a threshold were reached beyond 
which there were little or no measurable benefit to the patient. 


The Treatment Services Review (McLellan et ah, 1992a) provides a standard measure of the 
nature and number of services provided during the treatment process. Services are categorized 
along the same dimensions covered by the ASI and are recorded weekly. For the Minnesota 
OMS, the Treatment Services Record—a similar instrument—was developed (Harrison, in 
press). This instrument also uses the ASI dimensions to categorize services and quantifies hours 
in treatment and in specific components of treatment (educational sessions; individual, group, 
and family counseling sessions; support group meetings; and informal contact with other 
patients). Both the Treatment Services Review and the Treatment Services Record measure 
contacts that are part of formal treatment as well as ancillary services, since both may affect 
outcomes. The Iowa OMS also quantifies time involved in specific services, recorded on a 
monthly basis. 

Staffing Patterns 

Staffing patterns can be described by considering several objective and subjective qualities of 
staff members. These include the following: 








• Professional training and availability of staff with specialized training to do specialty work 

• Recovery status and prior AOD involvement of counseling staff 

• Therapist characteristics such as empathetic versus confrontational, directive versus facilitative 

• Management style 

• Staff morale 

• Amount of staff contact with patient. This contact may vary considerably in both duration and 
intensity. 

Few data exist on the relation of staffing patterns to patient outcomes. Selectivity in measuring 
the staff qualities listed above would be imperative if staff qualities were to be included as 
measures in an OMS. Most patients are likely to have contact with a variety of staff, so they may 
be exposed to a variety of levels of expertise, AOD use histories, and therapist characteristics. In 
addition, staff characteristics may not be static and may interact in different ways with individual 
patient characteristics. Although staffing characteristics and staffing patterns may be related to 
treatment outcomes, a decision might be made not to include these measures in an OMS because 
of lack of data about the benefits of collecting this information in relation to the time and costs 
involved. 

Context/Special Populations Served 

Programs vary tremendously in terms of the populations they serve. Special populations can be 
defined by personal characteristics such as gender, age, race/ethnicity, cultural background, or 
sexual orientation. Because of Federal block grant mandates, many States are currently designing 
more programs for pregnant women and women with children. Special populations can also be 
defined by drug use patterns; some programs are designed specifically for people addicted to 
heroin or cocaine. Programs can also be designed for individuals with coexisting disorders such 
as mental illness, eating disorders, or compulsive gambling. If an OMS were to find that certain 
patients did better in programs designed to serve their special needs, compared with other 
settings, these results could be used to justify increased funding for such targeted services, 
despite the increased costs these might entail. 

Discharge Status 

The patient's treatment completion status is essential information to collect in an OMS. 
Successful treatment completion can be used as an intennediary outcome measure (Wickizer et 
ah, 1994), whereas leaving against staff advice may predict a poorer posttreatment outcome. 
Other reasons for failure to complete treatment may provide useful information to improve 
treatment retention. 


The patient's treatment completion status is essential information to collect in an OMS. Successful 
treatment completion can be used as an intermediary outcome measure, whereas leaving against 
staff advice may predict a poorer posttreatment outcome. 








Patient Satisfaction 


Patient satisfaction with treatment as a whole or with specific services received can be measured 
during and after treatment. Satisfaction ratings by service type are included in the Minnesota 
weekly Treatment Services Record as well as in the followup interview. The Iowa system also 
includes patient ratings of the benefits of certain treatment components collected at followup. 
While patient ratings of treatment may not be related to other treatment outcomes, they may be 
important to consider in an era that is becoming more attuned to customer satisfaction. 
Satisfaction may also be related to treatment retention, so a correlation between patient 
dissatisfaction and early dropout from treatment may provide useful information about program 
changes necessary to keep patients in treatment. 

Posttreatment Services 

The use of community resources following discharge from treatment may also have an impact on 
patient outcomes. Patients are typically referred to peer support groups and other available 
resources to assist with their recovery following discharge from formal treatment. Learning more 
about the relation of these services and successful treatment outcomes can assist with discharge 
planning and resource allocation. 


Existing State Systems 

The content included in existing State outcomes monitoring systems provides a basis for review 
by States planning a new system or revising their current systems. Several State systems, in 
varying stages of development, are described in Appendix B. A contact resource is listed for 
each, to aid readers in obtaining a set of current instruments and other relevant information. 
Because of their length and because revisions occur periodically, individual forms are not 
reproduced in this TIP. 

All the discussion of OMS content in this chapter is based on experience to date. In this rapidly 
evolving field, recommendations will change as new information emerges from existing systems 
and ongoing studies. Some of the data elements currently in widespread use may be found to be 
of limited utility in evaluating outcomes. Others, in trial stages or not yet developed, may prove 
to offer more benefits. While this TIP can provide a useful overview of outcomes monitoring 
systems and general guidelines for content, decisions for an individual State will depend on local 
needs and an up-to-date review of continuing improvements in this field. 


Chapter 6—Legal Issues in Outcomes 
Monitoring 1 

Outcomes monitoring for alcohol and other drug (AOD) treatment is essential if advances are to 
be made. However, information about all persons receiving AOD abuse assessment, referral, and 
treatment services is protected by Federal laws and regulations (42 U.S.C.§§290dd-3 and ee-3 
and 42 Code of Federal Regulations, Part 2). 2 Designed to protect patients' privacy rights in order 
to encourage people to enter treatment, the Federal confidentiality laws and regulations 
concerning AOD-related information are more restrictive of communications in many instances 
than, for example, either the doctor-patient or the attorney-client privilege. Outcomes 
monitoring systems must be designed with these rules in mind. 

In addition, those designing outcomes monitoring systems that place burdens on AOD patients 
(for example, the burden of submitting to interviews) should obtain patients' informed consent to 
participate in the research. This kind of consent is different from the kind of patient consent that 
authorizes a program to release confidential information. In some outcomes monitoring systems, 
both kinds of consent will be called for. This chapter describes both kinds of patient consent. 

The larger portion of this chapter examines how, given the restrictions on disclosure of 
information about patients in treatment, outcomes monitors can gain access to sufficient 
information to evaluate AOD programs. Although most treatment providers are familiar with the 
Federal regulations protecting patient-identified information, the rules are worth reviewing in the 
outcomes monitoring context. For example, do outcomes monitoring systems incorporating 
followup interviewing of patients require that patients sign consent forms permitting the release 
of information? How can those monitoring outcomes track patients in other social service 
systems? These kinds of questions are addressed in this chapter. 

The chapter is divided into five parts. The first section outlines the scope of the Federal 
confidentiality laws and regulations. The rules governing researchers' access to information from 
AOD programs are described in the second section. The third section addresses the issues raised 
by research designs calling for followup with patients and with collateral sources, such as 
relatives, schools, and employers. Getting patients' informed consent to participate in outcomes 
monitoring systems that place a burden on them (such as submitting to interviews or pennitting 
contact with others) is also discussed. The third section also examines the special challenges 
raised by outcomes monitoring that seeks to track patients to see whether they reenter the 
treatment system, receive social welfare benefits, or have encounters with the criminal justice 
system. The final section addresses mandatory reporting laws and coding information about 
patients. 



Overview of the Federal Confidentiality Laws 

The Federal confidentiality laws and regulations prohibit disclosure of information about patients 
who have applied for or received any alcohol or drug abuse-related services—including 
assessment, diagnosis, counseling, group counseling, treatment, or referral for treatment—from a 
covered program. The restrictions on disclosure apply to any information that would identify a 
patient as an alcohol or drug abuser, either directly or by implication. The general rule applies 
from the time the patient makes an appointment. It also applies to patients who are civilly or 
involuntarily committed, minor patients, patients who are mandated into treatment by the 
criminal justice system, and former patients. The rule applies whether or not the person making 
an inquiry already has the information, has other ways of getting it has official status, is 
authorized by State law, or comes armed with a subpoena or search warrant. 

Any program that specializes, in whole or in part, in providing treatment, counseling and/or 
assessment, and referral services for patients with alcohol or drug problems must comply with 
the Federal confidentiality regulations (§§2.12(e)). The Federal regulations apply only to 
programs that receive Federal assistance, including indirect forms of Federal aid such as tax- 
exempt status, or State or local government funding coming (in whole or in part) from the 
Federal Government. 4 

Obtaining Information From AOD Programs 

The Federal confidentiality regulations provide three ways for researchers to obtain information 
from AOD programs: 

• The regulations permit AOD abuse treatment programs to give researchers access to 
information about patients when no patient-identifying information is revealed. 5 

• The regulations permit AOD programs to give researchers patient-identifying information 
without patients' consent when certain criteria are met. 

• Researchers may also obtain information that is protected by the Federal confidentiality 
regulations if patients sign proper consent forms. 

Access to data that do not contain patient-identifying information. The Federal regulations 
permit programs to disclose information about patients if the programs reveal no patient 
identifying information. "Patient-identifying information" is information that identifies an 
individual as an alcohol or drug abuser. Thus, a program can give researchers aggregate data 
about its population or some portion of its population. For example, a program staff member 
could tell a researcher engaged in outcomes monitoring that during the last year, 42 patients 
completed the treatment program, 67 dropped out in less than 6 months, and 25 left the program 
between 6 and 12 months. 

The research, audit, and evaluation exceptions. The confidentiality regulations permit 
programs to disclose patient-identifying information to researchers, auditors, and evaluators 
without patient consent, providing certain safeguards are met (§§2.52, 2.53). 


Research. AOD programs can disclose patient-identifying information to persons conducting 
"scientific research" if the program director determines that the researcher 1) is qualified to 
conduct the research, 2) has a protocol under which patient-identifying infonnation will be kept 
in accordance with the regulations’ security provisions (see §§2.16), 6 and 3) has provided a 
written statement from a group of three or more independent individuals who have reviewed the 
protocol and detennined that it protects patients' rights. 

Researchers are prohibited from identifying any individual patient in any report or otherwise 
disclosing any patient identities except back to the program. This provision is addressed more 
fully below, as it is particularly important when a research design calls for followup research 
with the patient or collateral sources or for tracking patients in other health, social welfare, or 
criminal justice systems. 

Audit and evaluation. Patient records may be reviewed on the program's premises for the 
purposes of conducting an audit or evaluation by the following entities: 

• Federal, State, and local government agencies that fund or are authorized to regulate a program 

• Private entities that fund or provide third-party payments to a program 

• Peer review entities performing utilization or quality control review in order to conduct an audit 
or evaluation. 7 

Any person or entity reviewing patient records to perform an audit or conduct an evaluation must 
agree in writing that it will use the information only to carry out the audit or evaluation and that 
it will redisclose patient information only 1) back to the program, 2) in accordance with a court 
order to investigate or prosecute the program (§§2.66), or 3) to a government agency overseeing 
a Medicare or Medicaid audit or evaluation (§§2.53(a), (c), (d)). Any other person or entity that 
is detennined by the program director to be qualified to conduct an audit or evaluation and that 
agrees in writing to abide by the restrictions on redisclosure can also review patient records. 
Again, the prohibition on redisclosure is particularly important when research designs include 
followup. 

When a researcher seeking to interview patients or fonner patients meets the requirements of 
§§2.52 or 2.53, the Federal confidentiality regulations do not require that a program obtain a 
patient's consent under §§2.31 to release his or her name to the researcher. However, it is always 
better practice to obtain patients' consent to the release of their names to researchers, auditors, or 
evaluators seeking to approach them for interviews. 

Consent 

Researchers can also obtain patient-identifying information if the patient has agreed to the 
release of the information by signing a valid consent form that has not expired or been revoked 
(§§2.31). The regulations' requirements regarding consent are somewhat unusual and strict and 
must be carefully followed. 


A proper consent form must be in writing and must contain each of the items contained in 
§§2.31: 



• The name or general description of the program or person making the disclosure 

• The name or title of the individual or organization that will receive the disclosure 

• The name of the patient who is the subject of the disclosure 

• The purpose or need for the disclosure 

• How much and what kind of information will be disclosed 

• A statement that the patient may revoke the consent at any time, except to the extent that the 
program or person authorized to make a disclosure has already acted in reliance on it 

• The date, event, or condition upon which the consent expires, if not previously revoked 

• The signature of the patient (and, in some States, his or her parent) 

• The date on which the consent is signed. 

A sample consent form is shown in Exhibit 6-1. A general medical release form or any consent 
form that does not contain all of the elements listed above is not acceptable. 

Several items on this list merit further explanation: 

The purpose of the disclosure and how much and what kind of information will be 
disclosed. These two items are closely related. All disclosures, and especially those made 
pursuant to a consent fonn, must be limited to information that is necessary to accomplish the 
need or purpose for the disclosure (§ §2.13(a)). If the recipient of the information needs only one 
specific piece of information, it would be improper to disclose anything more. 

In completing a consent fonn, it is important to determine the purpose or need for the 
communication of information. Once this need has been identified, it is easier to determine how 
much and what kind of information will be disclosed, tailoring it to the essentials needed to 
accomplish the identified need or purpose. 

For example, a researcher may want to study length of stay in different treatment programs. If 
the researcher decides to use consent forms to obtain pennission from patients to obtain this 
information from each program, he or she would specify the purpose of the disclosure as 
"verification of length of treatment" and the amount and kind of information to be disclosed as 
"enrollment and termination dates." The disclosure would then be limited to informing the 

researcher of the dates of enrollment and tennination for each patient. 

Another example involves a researcher who wants to study not only the length of stay but also 
the reasons for termination from treatment. The consent form would specify the purpose of the 
disclosure as "verification of length of treatment and reasons for tennination" and the amount 
and kind of information to be disclosed as "enrollment and termination dates and reasons for 
tennination." 

The patient's right to revoke consent. The patient may revoke consent at any time and the 
consent fonn must include a statement to this effect. Revocation need not be in writing. If a 
program has already given information to a research or evaluation entity prior to the revocation, 
the program has acted in reliance on the consent and is not required to try to retrieve the 
information it has already disclosed. 




Expiration of consent form. The form must also contain a date, event, or condition on which it 
will expire if not previously revoked. A consent must last "no longer than reasonably necessary 
to serve the purpose for which it is given" (§§2.31(a)(9)). The consent form does not need to 
contain a specific expiration date, but may instead specify an event or condition. 

The signature when the patient is a minor (and the issue of parental consent). A minor must 
always sign the consent form in order for a program to release information, even to his or her 
parent or guardian. The program must get the parent's signature in addition only if the program 
was required by State law to obtain parental permission before providing treatment to the minor 
(§§2.14). ("Parent" includes parent, guardian, or other person legally responsible for the minor.) 

In other words, if State law does not require the program to obtain parental consent in order to 
provide services to a minor, then parental consent is not required to make disclosures 
(§§2.14(b)). If State law requires parental consent in order to provide services to minors, then 
parental consent is required to make any disclosures. The program must always obtain the 
minor's consent for disclosures and cannot rely on the parent's signature alone. 

Required notice against redisclosing information. Persons who receive patient identifying 
information from AOD programs are strictly prohibited from redisclosing that information 
(§§2.32). However, patients may sign consent forms authorizing redisclosures. 


Exhibit 6-1 

Consent to Participate in Alcohol or Other Drug Abuse 
Treatment Research, Evaluation, and Followup 
Consent for the Release of Confidential Information 

I,_[name of patient], consent to 

participate in alcohol and other drug abuse treatment research, evaluation, and followup. I 
understand that I will be interviewed by 

_, a [clinician/ 

independent researcher/State employee], during my time in treatment about my problems 
and the services I receive. I understand that I will also be contacted and interviewed after 

I finish treatment by_, 

a [clinician/independent researcher/State employee] and asked about my progress since 
treatment. 

For purposes of participating in this research, I authorize 


[name of treatment program] to disclose my name to 

_[name of agency 

conducting the outcomes monitoring systems study] so that it can have someone 
interview me. 

I am providing an address and telephone number where I believe I can be located in the 












future, and the names, addresses, and phone numbers of others who may be of help in 
locating me. I understand that these persons will be contacted only concerning my 
whereabouts and that nothing about my treatment or my condition or the fact that I was in 
treatment will be disclosed to them or anyone else. 

I understand that my records and the information I provide are protected under the 
Federal regulations governing Confidentiality of Alcohol and Drug Abuse Patient 
Records, 42 CFR Part 2, and cannot be disclosed without my written consent unless 
otherwise provided for in the regulations. I also understand that I may revoke this consent 
at any time except to the extent that action has been taken in reliance on it, and that in any 
event this consent expires as follows: 


Specification of the date, event, or condition upon which this consent expires 
Dated: 


Signature of patient 


Signature of parent, guardian, or 
authorized 

representative when required 


Followup Research 

Research that follows patients for any period after they leave treatment presents special 
challenges. First, under the Federal regulations, no information the researcher or evaluator 
gained from the AOD program with the patient's consent or through the research, audit, and 
evaluation exceptions may be disclosed to anyone else. Yet, the researcher must locate the 
patient in order to collect followup data. Second, any research that can be perceived as imposing 
a burden on patients (for example, the burden of submitting to interviews) or that may require the 
researcher to contact either collateral sources or persons who can help locate patients in the 
future will require patients' informed consent to participate. This question is discussed first. 

Obtaining Informed Consent 

The best practice is to obtain patients' informed consent to participate in research that will 
impose a burden on them or that may require the researcher to contact others to inquire about the 
patient. This kind of consent is different from the consent discussed above, which authorizes an 
AOD program or researcher to disclose information to a third party. 












Obtaining informed consents from patients ensures that they agree to participate in the research 
project; that they understand that, as part of the project, efforts will be made to locate them 
through the persons whose names they submit (or that they or collateral sources will be 
interviewed); and that they appreciate the risks of participation (see Exhibit 6-2). In research that 
involves followup, the risk to the patient is most likely limited to a disclosure that he or she was 
in AOD treatment. Obtaining informed consent does not authorize a researcher to make a 
disclosure ofpatient- identifying information to third parties. As will be discussed below, if 
followup research is designed and conducted with care, it need not involve disclosures to third 
parties. If disclosure will be made to third parties, the researcher must obtain patients' consent in 
accordance with §§2.31. 

Conducting Followup Research 

Locating and interviewing former patients without making any prohibited disclosures. To 

ensure that patients can be located months or years after they leave treatment, researchers 
sometimes ask for the names of persons with whom the patients are likely to have continued 
contact. Making inquiries in order to locate a former patient might seem at first glance to pose no 
risk to a patient's right to confidentiality; nevertheless, it does. For example, if a research entity 
were to tell a landlord, either directly or by implication, that was looking for Marvin Moe in 
order to see how well he was doing after treatment, it would be letting the landlord know that 
Marvin Moe had been in treatment. The Federal regulations clearly prohibit this kind of 
disclosure, unless the patient consents. Thus, when researchers and evaluators are trying to locate 
a patient, they must do so without disclosing to others any infonnation about the patient's 
connection to AOD abuse treatment. 

If followup contact is to be attempted over the telephone, the caller has to be sure he or she is 
talking to the patient before identifying himself or herself or mentioning a connection to the 
AOD program. For example, asking for Sally Soe when her husband or child answers the phone 
and announcing that the caller is from the Capitol City AOD Program (or the Drug Research 
Corporation) violates the regulations. The program (or research agency) may form another entity, 
without a hint of drugs or alcohol in its name (for example, Health Research, Inc.), that can 
contact former patients without worrying about disclosing infonnation simply by giving its 
name. 

When a representative of such an entity calls former patients, however, he or she still has to be 
careful that the patients are personally on the line before revealing any connection to the AOD 
program. It is a good idea for the research entity to have a set of scripted answers the caller can 
give when questioned by others about the purpose of the inquiry about the fonner patient. 

If followup is to be done by mail, the return address should not disclose any information that 
could lead someone seeing the envelope to conclude that the former patient was in treatment. 

Followup with collateral sources. Outcomes monitoring that collects data about patients from 
collateral sources raises a similar issue to that raised by followup with patients themselves: How 
can an inquiry be made of relatives (including parents), employers, schools, or social welfare 
agencies without violating the Federal regulations? 



There are two ways to approach this problem. First, the outcomes monitor can structure the data 
gathering to avoid revealing that the patient was in AOD treatment. To accomplish this, the name 
of the entity that conducts the monitoring must be neutral, revealing nothing about an AOD 
connection. The questions asked of the collateral sources must also be drafted so that they offer 
those sources no information that would directly or implicitly link the patient with AOD abuse or 
treatment. 


Exhibit 6-2 

Consent to Participate in Alcohol and Other Drug Abuse 
Treatment Research, Evaluation, and Followup 

You are invited to participate in a study of alcohol and other drug abuse treatment effectiveness. 
The [name of agency conducting study] and treatment programs throughout the State want to 
learn more about what kinds of programs and services help different kinds of clients with 
different kinds of problems. 

What you are being asked to do . 

Each treatment program is asking at least [number of] clients to participate in this research. If 
you agree to participate, you will be interviewed by a [clinician/independent researcher] about 
the problems you have had and the kind of help you need. The first interview will take 30 to 45 
minutes. Then, each week you are in treatment, you will be interviewed for 5 to 10 minutes to 
record the services you are getting and your opinion of them. 

The final interview will take place 6 months after you leave treatment. You will be 
[telephoned/interviewed in person] by a [clinician/independent researcher/State employee] and 
asked about your progress since treatment. This followup interview will take 10 to 20 minutes. 

During the [first] interview, you will be asked some sensitive questions that might make you 
uncomfortable. You will also be asked about some acts that are illegal, such as drug use. You do 
not have to answer any questions you do not want to answer. However, your answers will be 
kept confidential. 

Other than the discomfort you may feel about some questions, there are no risks associated with 
your participation in this research. There are also no specific benefits for you if you participate, 
but your participation may help treatment programs improve their services. 

All information you provide is confidential . 

All information you provide for this research is confidential and is protected by Federal 
laws and regulations as well as by [name of State statute]. However, the researchers must 
report any abuse or neglect of children. [This requirement varies from State to State.] None of 
the interview forms will have your name on them. Only the consent form attached to this sheet 
will have your name on it, and this will be used only to contact you after treatment for the final 
interview. 









All other forms will use a confidential code number. All the information you provide will be 
stored in a locked [area/cabinet] at [location]. Then, it will be sent to [name of central data 
collection facility] and be entered in a computer. Your name will not be entered in the 
computer. Reports about this research will not identify any individuals in any way. The 
information will be used only to evaluate different kinds of treatment programs and different 
services clients receive. 

Participation is voluntary . 

Participation in this research is voluntary. If you decide to participate, you can change your mind 
at any time. If you decide not to participate, or if you start but change your mind, your decision 
will not affect your treatment in any way. 

If you have any questions about this research now or later, you may call: 

You will be given a copy of this form to keep. 


The second way an entity conducting outcomes monitoring can gather information from 
collateral sources is to obtain the patients' consent to disclose to the collateral source the fact that 
the patient was in AOD treatment. The special consent form required by §§2.31 of the 
regulations must be used to obtain the patient's consent. As outlined above, this fonn requires the 
research entity to state on the consent form the purpose of the disclosure —in this instance, 
outcomes monitoring research—and how much and what kind of information will be disclosed, in 
this instance, the fact that the patient was in AOD treatment. The form must also have an 
expiration date and a statement that consent can be revoked at any time. 

Using a consent form to gather information from collateral sources may require more work 
initially—to obtain consent forms from all patients—but it provides more freedom to the 
researcher. With consent forms from patients, the researcher may ask questions about AOD use. 
However, care must still be taken to reveal only the limited information specified on the consent 
form. The researcher should have a system to keep track of the expiration dates of the consent 
forms being used. 

Tracking patients in social welfare and criminal justice systems. In some cases, outcomes 
monitors will want to detennine, by tracking patients, whether patients reenter the treatment 
system or receive other medical or social welfare benefits or have encounters with the criminal 
justice system. Because of the Federal confidentiality laws and regulations, special care must be 
taken in the design of any research in which an attempt is made to compare the names of patients 
who are receiving or have received AOD treatment with 1) the names of patients who previously 
received AOD treatment or who subsequently enter the treatment system, 2) the names of 
persons receiving other medical or social benefits, or 3) the names of persons arrested for 
criminal activity. 

Clearly, the entity conducting outcomes monitoring cannot simply turn over the names of the 
patients of the programs it is evaluating to other AOD programs, medical or welfare authorities, 








or officials in the criminal justice system with a request that a search be made for the names. 
How, then, can these tasks be accomplished? 

Tracking patients in the AOD system. Tracking whether or not patients reenter the treatment 
system generally requires the creation and continual updating of a database of patients’ names. 
Over the course of the study, researchers compare the names submitted by treatment programs to 
the names already in the database. This kind of study can be accomplished in any one of three 
ways: 

• Using patients' consents. All patients can sign consent forms that comply with §§2.31, 
permitting programs to disclose patient identifying information to researchers. The consent 
forms must specify the reason for the disclosure, what information will be disclosed to 
researchers, and an expiration date. (Note that this simple consent form does not permit the 
entity operating the database to disclose information to a treatment program or to anyone 
else.) 

• Using the research, audit, and evaluation exception. If the entity conducting the outcome 
monitoring satisfies the requirements of §§2.52 or 2.53, programs can disclose patient- 
identifying information for entry into and comparison with a database of patients' names. Again, 
no information from one treatment program can be disclosed to another. 

• Using the gualifiedservice organization exception. The Federal confidentiality regulations permit 
AOD programs to enter into an agreement, called a qualified service organization agreement 
(QSOA), when they routinely need to share certain information with an outside agency that 
provides services to them. A QSOA is a written agreement between a program and a person or 
entity providing services to the program, in which that person or entity does the following 
(§§2.11, 2.12(c)(4)): 

^Acknowledges that in receiving, storing, processing, or otherwise dealing with any 
patient records from a program, it is fully bound by [the Federal confidentiality] 
regulations 

2)Promises that, if necessary, it will resist injudicial proceedings any efforts to obtain 
access to patient records except as pennitted by these regulations. 

An entity performing outcomes monitoring is providing a service to the program by studying 
how effective treatment is and how it might be improved. Disclosures made to an entity 
conducting outcomes monitoring with which programs have signed QSOAs must be limited to 
information that is needed to conduct the research. (See sample QSOA form in Exhibit 6-3.) 

• Using the internal communications exception. The Federal regulations permit programs to 
disclose information to an entity that has direct administrative control over them if that entity 
needs "the information in connection with [its] duties that arise out of the provision of 
diagnosis, treatment or referral for treatment of alcohol or drug abuse" (§§2.12(c)(3).). If the 
entity conducting the outcomes monitoring is the single State agency (SSA), the programs it runs 
could disclose the information it needs in order to conduct that monitoring under this 
exception, without patients' consent. Flowever, the SSA could not redisciose patient-identifying 
information to anyone else. 



Exhibit 6-3 

Qualified Service Organization Agreement 

XYZ Service Center ("the Center") and 
the 


(name of the program) 

("the Program") hereby enter into a qualified service organization agreement, whereby 
the Center agrees to provide the following services: 


(nature of services to be provided) 


Furthermore, the Center: 

1. Acknowledges that in receiving, storing, processing, or otherwise dealing with any 
information from the Program about the patients in the Program, the Center is fully 
bound by the provisions of the Federal regulations governing Confidentiality of Alcohol 
and Drug Abuse Patient Records, 42 CFR Part 2: and 

2. Undertakes to resist injudicial proceedings any effort to obtain access to information 
pertaining to patients otherwise than as expressly provided for in the Federal 
confidentiality regulations, 42 CFR Part 2. 

Executed this_day of_, 199_. 


President Program Director 

XYZ Service Center (name of program) 

(address) (address) 


Tracking patients in health, social welfare, and criminal justice systems. The Federal 
confidentiality regulations' restrictions on the disclosure and redisclosure of patient-identifying 
in formation generally make it difficult to track whether or not patients seek treatment from other 
medical or mental healthcare providers, apply for welfare benefits, or are arrested by law 
enforcement authorities. As noted above, the entity conducting outcome monitoring cannot 
simply turn over the names in its database to medical or welfare authorities or to officials of the 
















criminal justice system, since the Federal confidentiality regulations prohibit this kind of 
disclosure. How can an entity conducting outcomes monitoring compare a database of patients' 
names with a similar database created by the healthcare, welfare, or criminal justice systems? 

A comparison of the names of patients in AOD programs with the names of patients receiving 
other medical or mental healthcare or welfare benefits can be accomplished in one of two ways: 

• If the entity conducting the outcomes monitoring is an SS/\ that also administers programs and 
has jurisdiction over mental and mental healthcare and/or welfare. 

As noted above, the Federal regulations permit programs to disclose information to an entity that 
has direct administrative control over them if that entity needs "the information in connection 
with [its] duties that arise out of the provision of diagnosis, treatment, or referral for treatment of 
alcohol or drug abuse" (§§2.12(c)(3)). If the entity conducting the outcomes monitoring is the 
SSA that runs AOD programs and also has jurisdiction over medical and mental healthcare, the 
AOD, medical, and mental healthcare providers under its jurisdiction could disclose the 
information the SSA needs in order to conduct outcomes monitoring under this exception. 
However, the SSA must resist the temptation to use this data for another purpose. For example, it 
could not redisclose the infonnation (names) it receives from AOD treatment providers to other 
AOD treatment providers or to mental healthcare providers or welfare authorities. 

• If the entity conducting the outcome monitoring is an SS/\ with jurisdiction solely over AOD 
treatment or is any other organization. 

Using patients' consents. All AOD patients can sign consent forms that comply with §§2.31, 
pennitting programs to disclose patient-identifying information to the entity conducting the 
research. The consent forms must specify the reason for the disclosure, what information will be 
disclosed to researchers, and an expiration date. Patients receiving care from other medical or 
mental healthcare providers can sign appropriate consent fonns permitting those providers to 
disclose their names to the entity conducting the outcomes monitoring. No patient-identifying 
information that entity receives from AOD treatment programs can be redisclosed. 

Using the research, audit, and evaluation exception for AOD information and patients' 
consents for other information. If the entity conducting the outcomes monitoring satisfies the 
requirements of §§2.52 or 2.53, programs can disclose patient-identifying information for entry 
into and comparison with a database of patients’ names submitted by other medical and mental 
healthcare providers with patient consent. 

Using the qualified service organization exception for AOD information and patients' 
consents for other information. The entity conducting outcomes monitoring could collect 
information from AOD programs by using a QSOA, as outlined above, and from medical care, 
mental health, and welfare authorities by using appropriate patient consent. 

Tracking patients through the criminal justice system is particularly difficult. While an entity 
conducting outcomes monitoring can use any one of the mechanisms listed here to obtain the 




names of AOD patients, it may have difficulty obtaining information from the criminal justice 
system because of State laws protecting the confidentiality of that information. 


Other Issues 

Research and Mandatory Reporting Laws 

If a researcher uncovers child abuse or neglect while reviewing a patient's chart or interviewing 
the patient, must he or she make a report to State authorities? How can a report be made under 
the Federal confidentiality laws and regulations? 

Is the researcher required to report? All 50 States have statutes requiring reporting when there 
is reasonable cause to believe or suspect child abuse or neglect. While many State statutes are 
similar, each has different rules about what kinds of conditions must be reported, 9 who must 
report, and when and how reports must be made. Most States now require not only physicians 
but also educators and social service workers to report child abuse. What is unclear, however, is 
whether or not persons conducting research are mandated reporters. 10 This question is important 
because, as noted below, only those who are mandated by State law to report child abuse and 
neglect may do so under the Federal laws and regulations. 

Because of the variation in State laws, researchers who are concerned about this issue should 
consult an attorney familiar with State law to determine whether they are mandated to report 
child abuse. 

How can a researcher make a report? In 1986, the Federal confidentiality regulations were 
amended to permit AOD programs to comply with mandatory child abuse reporting laws. The 
Federal laws and regulations now permit compliance with State laws that require the reporting of 
child abuse and neglect. However, this exception to the general rule prohibiting disclosure of any 
information about a patient applies only to initial reports of child abuse or neglect. Programs and 
others cannot respond to followup requests for information or even subpoenas for additional 
information, even if the records are sought for use in civil or criminal proceedings resulting from 
the researcher's initial report, unless the patient consents or the appropriate court issues a special 
court order (see §§2.64 and 2.65). 

Any researcher contemplating making a child abuse report would be wise to check with the 
program to see if a report has already been made. 

Coding Patients' Identities 

If a researcher codes patients' names to protect their identities, can the intricate rules of the 
Federal confidentiality laws and regulations be disregarded? It depends. As noted above, the 
Federal regulations protect "patient-identifying information." Section 2.11 of the regulations 
defines this to mean the name, address, social security number, fingerprints, photograph, or 
similar information by which the identity of a patient can be detennined with reasonable 


accuracy and speed either directly or by reference to other publicly available information. The 
tenn does not include a number assigned to a patient by a program, if that number does not 
consist of or contain numbers (such as a social security or a driver's license number) that could 
be used to identify a patient with reasonable accuracy and speed from sources external to the 
program. 

Thus, if a research entity can truly code a patient's name so that the number created for each 
patient cannot be "used to identify a patient with reasonable accuracy and speed from sources 
external to the program," it need no longer be concerned with safeguarding infonnation about the 
patient. 


Conclusion 

Outcomes monitoring systems must be designed with patients' interests in mind. Since both 
informed consent and issues of confidentiality can be central to research design, researchers 
should seek the advice of an attorney familiar with these issues to ensure that patients' rights are 
protected. 


Endnotes 

1. This chapter was written for the consensus panel by Margaret K. Brooks, Esq. The Center for 
Substance Abuse Treatment has published another document Confidentiality of Patient Records 
for Alcohol and Other Drug Treatment (Technical Assistance Publication Series 13) that 
addresses these issues. 

2. Hereinafter, citations in this section in the form "§§2..." refer to specific sections of 42 C.F.R., 
Part 2, implementing the Comprehensive Alcohol Abuse and Alcoholism Prevention, Treatment 
and Rehabilitation Act of 1970 (42 U.S.C. §§290dd-3) and the Drug Abuse Prevention, 
Treatment and Rehabilitation Act (42 U.S.C. §§290ee-3). 

3. Violating the regulations by disclosing information is punishable by a fine of up to $500 for a 
first offense or up to $5,000 for each subsequent offense (§§2.4). 

4. Two additional Federal laws permit the U.S. Attorney General and the Secretary of the U.S. 
Department of Health and Human Services (DHHS) to authorize researchers to withhold the 
names and identities of research subjects. The statute authorizing the Secretary of HHS to issue 
confidentiality certificates specifies that it applies to "persons engaged in biomedical, behavioral, 
clinical, or other research (including research on mental health, including research on the use and 
effect of alcohol and other psychoactive drugs)...." Once such authorization is issued, the 
researcher "may not be compelled in any Federal, State or local civil, criminal, administrative, 
legislative, or other proceeding to identify the subjects of research for which such authorization 
was obtained," (42 U.S.C. §§241 (d) permits the Secretary of HHS to issue confidentiality 


certificates; 21 U.S.C. §§872(c) permits the U.S. Attorney General to issue confidentiality 
certificates). 


These statutes may not be particularly useful for those conducting outcomes monitoring because 
most of the infonnation they will gather will already be protected by the Federal confidentiality 
regulations. However, depending upon who gathers followup information from patients, it is 
possible that this information will not be protected by 42 C.F.R., Part 2, as it could be argued that 
the former patients are making voluntary disclosures of information about themselves directly to 
data gatherers. As with many questions about the design of outcomes monitoring, it is best to 
check with an attorney familiar with Federal and State laws and regulations for guidance. 

5. "Patient-identifying information means the name, address, social security number, 
fingerprints, photograph, or similar information by which the identity of a patient can be 
determined with reasonable accuracy and speed either directly or by reference to other publicly 
available information. The term does not include a number assigned to a patient by a program, if 
that number does not consist of or contain numbers (such as social security, or driver's license 
number) which could be used to identify a patient with reasonable accuracy and speed from 
sources external to the program" (§§2.11). 

6. Section 2.16 requires programs to keep written records in a secure room, a locked file cabinet, 
a safe, or other similar container. The program should have written procedures that regulate 
access to and use of patients' records. Either the program director or a single staff person should 
be designated to process inquiries and requests for infonnation. 

7. These particular entities may also copy or remove records, but only if they agree in writing to 
maintain patient-identifying information in accordance with the regulations' security 
requirements (see §§2.16), to destroy all patient-identifying information when the audit or 
evaluation is completed, and to redisclose patient information only 1) back to the program, 2) in 
accordance with a court order to investigate or prosecute the program (§§2.66), or 3) to a 
government agency overseeing a Medicare or Medicaid audit or evaluation (§§2.53(b). 

8. For a discussion of this issue in the context of a consent form permitting interviews with 
collateral sources, see the discussion on "Followup with collateral sources." 

9. For instance, in some States, certain professionals are required to report pregnant women who 
abuse drugs to State authorities. If such a requirement is part of the State's mandatory child abuse 
reporting law, the Federal regulations permit the report. If a separate statute mandates the report, 
the Federal regulations probably do not permit the report. 

10. Some State statutes are worded to require reports only from those persons who have face-to- 
face interaction with a child or adult who discloses abuse. 



Chapter 7—OMS Budget Considerations 

A variety of factors will influence the costs of an outcomes monitoring system (OMS) and these 
are worth considering during the planning phase. All single State agencies (SSAs) will be 
challenged to find a compromise between the ideal design and what they can afford to spend. 


Personnel/Contracted Services 

From the initial design stages of an OMS, a partnership must be forged among the SSA, 
independent researchers, and alcohol and other drug abuse (AOD) abuse treatment providers, 
with the single State agency and researchers providing primary oversight for the process. This 
same partnership should continue as the design is implemented and data collection efforts begin. 
However, there are expenses associated with the involvement of these various personnel. 

Steering Committee and Working Groups 

As described in Chapter 2, the support and involvement of a variety of stakeholders are essential 
to the successful design and implementation of an OMS. The State agency must consider to what 
extent expenses associated with the steering committee or workgroups will be borne by the 
participants or by the State. Expenses will include the cost of meeting rooms, refreshments, 
mileage reimbursements and other travel costs, and overnight lodging for those who travel a 
great distance. 

Participants' time can be used most efficiently if meeting agendas are well planned and 
objectives for each discussion are clear. Participants can be mailed drafts of instruments and 
design plans in advance of meetings to maximize efficient use of meeting time. Some tasks can 
be assigned to small workgroups, and they can report on these to the whole group. For some 
decisions, such as approval of interim or final instrument or report drafts, reading the mailed 
materials may be just as productive as attending a meeting. In some areas, telephone conferences 
or videoconferencing may save costs, particularly when many participants must travel a great 
distance to the meeting site. 

Expert Researchers 

The role of independent expert researchers in developing and implementing an OMS can be 
significant. Outside academic researchers are subject to peer review standards and are likely to 
be able to create a rigorous data collection design. Their input is particularly important at two 
points: in the early design and implementation phases of the system, when they help ensure the 
integrity and credibility of the data, and in the data analysis phase of the project, when they can 
help interpret results. Another advantage of hiring independent researchers is that they are 
unlikely to have conflicts with any of the interests involved. 



However, State agencies lacking the resources to hire independent researchers should not view 
this lack of resources as an obstacle to the successful implementation of an OMS. This three-way 
collaboration among outside experts, treatment providers, and State personnel may not be the 
answer for all States and systems. Some State agency managers may be reluctant to relinquish 
control of their OMS to an external source. In some instances, sufficient research expertise exists 
within the single State agency to design and implement an effective OMS without hiring outside 
experts, and additional staff can be added as needed. The possibility of conflict of interest does 
not warrant an expensive duplication of resources and can often be addressed with some external 
oversight procedure. 


In some instances, sufficient research expertise exists within the single State agency to design and 
implement an effective OMS without hiring outside experts, and additional staff can be added as 
needed. 


Project Staff 

Whether the project is based at the SSA or contracted to a research agency, a full-time project 
manager is essential. In addition, depending on the scope of the project, one or more of the 
following staff will also be needed: project secretaries, systems analysts, programmers, and data 
clerks. Research analysts with statistical expertise will also be needed for monitoring data 
quality, analyzing results, and writing reports and presentations. 

Training 

Treatment program staff can collect patient admission and treatment data more efficiently than 
anyone else. However, specialized training will usually be necessary, and major costs will be 
related to staff training. Training sessions can be conducted by SSA staff or outside contractors. 
Treatment providers can be trained in groups of 25-30 personnel at a time. It is recommended 
that each program send at least two staff to training—including those responsible for 
administering structured interviews and fdling out forms. Depending on the complexity of the 
data collection instruments, training sessions will probably last 1 to 2 days. While a detailed 
forms completion guide will be essential, there is no adequate substitute for in-person training. 
Whether travel expenses for program staff will be borne by the programs themselves or 
reimbursed by the SSA must be taken into account when setting up the training budget. 


Other Administrative Expenses 


Effects of Sample Size 







The single biggest driver of OMS project costs will be sample size. In fact, the availability or 
limits of funding may detennine sample size. The largest cost directly associated with sample 
size is followup interview costs, which are discussed in greater detail later in this chapter. 
However, the sample size will also detennine both the number of staff necessary to perform data 
entry and the printing costs for forms. 

Instrument Selection and Design 

As discussed in Chapters 4 and 5, instrument design is an expensive project in itself. New 
instruments require expert consultation, field tests, extensive data analyses, and refinements. To 
assure validity, they must be tested on the populations in which they will eventually be used. The 
need to design new instruments may delay the startup of an OMS. Because of the high costs 
associated with instrument design, the use of existing instruments is recommended. When these 
are inadequate for a State agency's needs, modifications will usually be cheaper than starting 
from scratch. 

Existing instruments developed with funding through public dollars are in the public domain. 
This classification means that no costs are associated with using the content of the instrument. 
However, there will be costs associated with duplicating an instrument or refonnatting it and 
printing it for project specifications. Software programs have been developed for some 
instruments in the public domain, and these are likely to have a licensing fee. Existing software 
may serve a State agency's needs and may justify the cost. 

Copyrighted instruments are usually sold on a per-unit basis and may cost up to several dollars 
per copy. For large quantities, it may be possible to negotiate a user's fee with the copyright 
holder rather than having to pay on a per-unit basis. Copyrighted instruments may not be 
modified without the copyright holder's pennission. 


The single biggest driver of OMS project costs will be sample size. In fact, the availability or limits of 
funding may determine sample size. 


Miscellaneous Expenses 

There are a number of other expenses that will have to be included in the OMS budget. These 
include costs for communications (telephone and mailing costs); meeting rooms and presentation 
equipment; costs for travel and accommodations, printing of forms, training manuals and reports; 
and supplies, including graphics software, training and presentation materials, and basic office 
supplies. 







Patient Followup 


Patient followup can be conducted through telephone interviews, mailed questionnaires, or in- 
person interviews. Mailed questionnaires typically have an unacceptably low return rate and thus 
are not recommended as the primary contact method for an OMS. While in-person interviews 
may be the ideal, the costs are very high. Patients are unlikely to follow through on a visit to an 
interview site without an incentive. Having interviewers visit patients at home can cost between 
$50 and $100 per patient. Telephone interviews may offer the best compromise. Costs will vary, 
depending on area of the country and length of interview. In Minnesota, a private research 
agency is conducting 6-month followup telephone interviews for the OMS at an average cost of 
$20 per patient. Bids for this project, however, ranged up to $40 per patient. 

While using treatment program staff to conduct followup interviews eliminates these costs, it is 
not recommended because of the burden imposed on staff and the potential threats to validity of 
data. Research agencies also have more experience locating respondents and conducting 
telephone interviews. 


While using treatment program staff to conduct followup interviews eliminates these costs, it is not 
recommended because of the burden imposed on staff and the potential threats to validity of data. 


Followup costs are directly associated with the number of contacts attempted. Conducting 
followups at both 6 and 12 months posttreatment will double the costs of 6-month interviews 
only. With a limited budget, it will be necessary to weigh the advantages of a larger sample that 
is contacted once versus that of a sample half the size that is contacted twice. 


Management Information Systems 

Hardware and Software 

When developing an outcomes monitoring system, State agencies must address issues related to 
computer technology. Many factors come into play, including the availability of existing 
resources. The primary focus of OMS development should not be hardware and software needs 
but rather the development of a well-planned system to meet the needs of the users. As system 
requirements are defined, answers to questions regarding appropriate computer software and 
hardware will become more apparent. The goal is to develop a system for gathering the required 
data while minimizing the number of intervening tasks that raise costs and increase the chance of 
error. 

Once system requirements are defined, SSA managers will need to consider the design in light of 
available funding and other resources. 







Interactive Data Collection Systems 


The ideal, most cost-effective system is a computerized interactive data collection system. Such 
a system allows staff at treatment centers to enter data directly into the OMS during patient 
interviews. The system would be designed to alert the interviewer immediately to inconsistent 
data or unacceptable responses and would allow the staff to resolve data errors as they are 
identified. Online data entry significantly improves data validity and greatly reduces the number 
of intervening tasks that must be performed by the State agency. Washington State has made 
great strides with the use of online data entry. 

Given the current cost and power of microcomputer technology, the hardware cost for a single 
microcomputer workstation is approximately $2,000, depending on the manufacturer and model. 
In large treatment centers where more than one microcomputer workstation would be required, a 
local area network (LAN) would have to be installed so that all workstations could share a 
common set of computer software and patient databases from a central computer or file server. 
The costs associated with a LAN vary greatly depending on the number of workstations, the 
distances between them, and the size of the network file server. A small LAN with one file 
server, serving fewer than 10 workstations in one location, would cost approximately $15,000. 

Software to support an interactive system can be purchased from a software vendor, provided the 
software complies with system requirements, or new software can be developed by either 
inhouse programming staff or an outside software programming firm. For example, ASI software 
is available. The costs associated with each approach may vary greatly. Purchasing existing 
software is usually less expensive than developing new software. Prepackaged software is 
usually purchased on a per-user basis, and for assessment/testing software there may also be a 
per-issuance charge. When purchasing software, the buyer usually is also purchasing the right to 
use the software. Therefore, the total number of treatment centers using the software will have a 
direct impact on the costs associated with its purchase. Some software vendors may be willing to 
modify their software in order to bring it into compliance with a State's systems requirements. 
However, the charges associated with the software modification plus the normal purchase price 
may exceed the cost of developing new software. 


Some software vendors may be willing to modify their software in order to bring it into compliance 
with a State's systems requirements. 


Other Data Entry Methods 

Data entry methods include keyboard entry, optical scanning, and online. With forms for 
keyboard data entry, answers are handwritten in blocks or boxes, and data entry personnel key in 
the responses. This method is the most labor intensive because of the extensive data entry 







required. Since newer, more cost-efficient technologies are available, this method is not 
recommended. 

Optical scannable forms have "bubbles" or circles to be filled in; these forms are "read" by a 
scanner and require less staff labor. An optical mark scanning system is a relatively inexpensive 
method for converting data collection forms into electronic data. Scanners can effectively 
process approximately 2,000 forms per hour, depending on the manufacturer. Scanners are 
usually connected to a microcomputer, using a serial interface cable. Software provided by the 
manufacturer handles the communications between the computer and the scanner. The software 
is also capable of performing basic editing on individual data fields and is able to alert the data 
clerk to an identified error. However, more complex editing—such as cross-field validations— 
would have to be perfonned using other software programs. The components needed to 
implement an optical mark scanning system cost approximately $12,000, depending on the 
hardware and software configuration. Data collection forms especially designed and printed for 
optical mark scanning cost approximately 5 cents per form, depending on the amount of time 
spent designing the form and quantity of forms being printed. 


Final Comments 

Because of the great variety of local needs and existing capacities, it is impossible to adequately 
address all budgetary considerations in a brief chapter. As State agencies initiate the OMS 
planning process, they are advised to consult with agencies in States that already have outcomes 
monitoring systems in operation. They can also consult with researchers who have experience 
with treatment outcome studies. While aiming for cost-efficiency is always a worthy endeavor, it 
must be accepted that valid outcomes monitoring will entail substantial costs. The key is to 
ensure that the ultimate utility and benefits of the OMS justify the investment. 


Chapter 8—Using the Results of Outcomes 
Monitoring 

Outcomes monitoring is meaningful only if the information it produces is put to good use. A 
major responsibility of the single State agency (SSA) in implementing an outcomes monitoring 
system (OMS) is to analyze, interpret, and disseminate OMS data in ways that will satisfy the 
needs of potential users. In addition, the SSA must be able to use the results of outcomes 
monitoring for its own purposes, including planning and policymaking. This chapter discusses 
some of the issues related to this dual responsibility for the uses of outcomes monitoring data. 


Promoting the Use of Outcomes 
Monitoring Data: Responsibilities of 
the Single State Agency 

• Analysis and interpretation 

• Translation and packaging for different users 

• Marketing of findings 

• Technical assistance to programs in using data. 


Who Uses Outcomes Monitoring Data, and How? 

The potential users of outcomes monitoring data include most of those organizations and groups 
identified in Chapter 2 of this Treatment Improvement Protocol (TIP) as important to the 
planning and development process of the OMS. Treatment providers will also have been 
involved in the process of data collection. These data will be forwarded to the SSA for 
compilation and interpretation, and the findings will be formatted and disseminated to the 
providers as well as various other users. 

In addition to the SSA and providers of treatment services, other users of OMS data may include 
the governor's office, the legislature, local or regional planning areas, and other health and 
human service agencies. Policy decisions made by these users on the basis of outcomes 
monitoring data will ultimately have an impact on the funding of programs and services to 
consumers. In this way, treatment outcomes data filter up and down within the State structure for 
use in different ways. 

Treatment programs will use data for the following: 


Accreditation 








• Program improvement 

• Program development 

• Program management. 

Local regional planning agencies and administrative units may use data for the following: 

• Allocate funding 

• Provide quality assurance 

• Plan for service delivery 

• License and accredit providers 

• Monitor programs 

• Assess community impact 

• Share with other health and human services agencies within their administrative units. 
Internal data is used by the SSA: 

• Development of policy recommendations and directions 

• Advocacy 

• Financial planning 

• Resource allocation 

• Identification of research areas 

• Trend analysis 

• Projections. 


Data Analysis, Interpretation, and Reporting 

A well-designed OMS will require adequate attention to data analysis. Data analysis should be 
performed by personnel who have the appropriate statistical training and expertise. The data 
analysis plan should be designed to answer the questions identified by OMS planners as most 
important for their audiences. 

Determining treatment outcomes involves analysis of patient characteristics and services 
received by the patient along with outcome measures. Adjunct services outside the AOD 
treatment system may also have an impact on outcomes and should be included in the analysis. 
The multivariate analyses required will be quite complex. The improper application of statistical 
procedures or unwarranted interpretations of statistical results can result in inaccurate 
conclusions, potentially jeopardizing the integrity, acceptance, and usefulness of the OMS. 


The improper application of statistical procedures or unwarranted interpretations of statistical results 
can result in inaccurate conclusions, potentially jeopardizing the integrity, acceptance, and usefulness 
of the OMS. 









The most often heard complaint about large data systems is that the data go in but never seem to 
come out. Unfortunately, this criticism is often justified. Frequently, the bulk of resources goes 
into data collection, and there are not enough funds and staff available for data analysis and 
reporting of results. 

Since multiple stakeholders will have different interests, the data analyses should be conducted 
to meet those multiple needs whenever possible. It is also important that the results of the 
analyses be communicated to interested parties in a timely, meaningful, and relevant manner. 

In order for an OMS to be useful and to obtain substantial buy-in and participation, OMS results 
must be made available on a timely and regular basis. Thus, it helps to have automated reports. A 
schedule of reports should be developed and disseminated to OMS participants and other 
interested persons. Guidelines for data delivery should be established as part of the OMS design 
parameters, just as data delivery guidelines from providers are incorporated into the system 
design. 

A balance must be sought between the demand for quick results and the realities of analyzing 
data and preparing reports. Some audience needs can be accommodated by designing a series of 
reports that release findings incrementally. For example, program participants could get 
quarterly, semiannual, and/or annual summaries of patient characteristics at admission. Tabular 
summaries can be programmed so that automated reports are generated at predetermined 
intervals. Washington State and Minnesota have both automated reports. Data can be presented 
program-by-program or aggregated statewide or by county. Data can also be aggregated by 
program type (for example, inpatient or outpatient). 

Programs can also obtain patient discharge summaries on a regular basis, including such data as 
program completion rates, payment source, and length of stay. Admission and discharge 
summaries can be extremely useful for planning purposes. Such patient profiles can help 
determine whether programs are serving their target populations, whether patterns exist in 
retention or dropout rates, and whether services delivered are appropriate to the needs described 
by patients. Regular data summaries are also useful for general audit purposes. 

Treatment service data are likely to be more complicated to analyze and present and may require 
a more conservative report schedule. Followup data demand the greatest lag time of all. Suppose, 
for example, that designers of an OMS wanted to include data on 1996 admissions, and the 
design included a 6-month followup. Programs would start collecting data in January 1996. 
Depending on the length of treatment, patients admitted in December 1996 might not be 
discharged until May 1997. Their followup interviews would be conducted in November or 
December 1997. By then, almost 2 years would have elapsed since the beginning of data 
collection before followup data were even ready to be analyzed. Realistically, at least 6 more 
months might be required for analysis and reporting. 

However, it may not be necessary to wait for followup to be completed on the entire year's 
admissions to begin to generate followup results. If the sample size is very large, preliminary 
results might be reported—for instance, admissions in the first quarter or first 6 months. 



Nonetheless, it is important that all personnel involved with analysis of OMS data understand 
that posttreatment followup, by its nature, extends the data collection and reporting process. 

Comprehensive results of the OMS should be presented in a document that describes data 
collection methods, limitations of the data, analyses, findings, and recommendations. Tennessee 
and Colorado have created such documents. In addition, more selective reports, tailored to 
specific users, should be prepared. A brief summary of major findings may be appropriate for 
policymakers. Treatment providers will be interested in the benefits to them, and the data must 
be transmitted in a fonn that allows practical application by program staff. Media outlets can be 
sent press releases and charts or graphs of findings. 


Comprehensive results of the OMS should be presented in a document that describes data collection 
methods, limitations of the data, analyses, findings, and recommendations. In addition, more 
selective reports, tailored to specific users, should be prepared. 


When the documentation of the OMS is developed, all participating groups should be invited to 
review and comment. Preliminary findings that may have important implications for these 
groups should be shared when appropriate. Some groups may want to develop their own action 
plans based on the findings. They may also want to develop a consensus statement in support of 
the findings. To encourage this kind of support, the groups should have the opportunity to 
comment before the final report and to find commonalities around which consensus can be 
developed. As a result, areas on which consensus is reached may move up in terms of their 
priority. 

If findings do not win acceptance with one or more groups, the single State agency should 
conduct an analysis to determine why. Does the lack of acceptance reflect reservations about the 
validity of the findings, or is it the result of political differences? The explanation for any failure 
to endorse OMS findings has important ramifications for how the SSA will want to respond. 

In preparing OMS findings for dissemination, and in all efforts to gain support for 
recommendations based on those findings, the emphasis must always be on improving outcomes 
for patients, their families, and society. This underlying principle should guide all policy 
decisions based on outcomes monitoring. 


In preparing OMS findings for dissemination, and in all efforts to gain support for recommendations 
based on those findings, the emphasis must always be on improving outcomes for patients, their 
families, and society. 















Setting Priorities for Needed Policy Changes 


A wide range of policy changes could be indicated by the findings of outcomes monitoring: 

• Developing or refining uniform patient placement criteria 

• Changes in funding targets, including reallocation from one program to another or one type of 
service to another 

• Changes in legislation (for example, to obtain new authorities for the SSA, codify interagency 
collaborations, or revise licensing or credentialing requirements) 

• Changes in rules and regulations that affect the way in which legislative mandates are 
interpreted and implemented 

• Changes in SSA administrative policies and practices, ranging from hierarchical lines of authority 
and departmental requirements to planning documents, interagency cooperative agreements, 
or memos of understanding 

• Changes in the future design of the OMS itself. 

A review of the hypothetical findings described in Chapter 1 and the examples displayed here 
will be helpful in illustrating the kinds of treatment system changes that might be envisioned. 

Most policy changes have important budgetary implications that must be kept in mind when 
setting priorities for policy change. Any efforts to bring about policy change should be 
undertaken only after careful assessment of the budgetary implications and the development of 
realistic proposals for addressing them. A policy mandate that does not carry with it the 
assurance of additional monies will necessitate reallocation of funds from other areas. Cost¬ 
sharing with other agencies may need to be developed through interagency agreements. 


Implementing Policy Change 

Working With State Policymakers 

Implementation of desired policy changes identified through outcomes monitoring will require 
effective working relationships among policymakers at various levels within the State. If these 
policymakers were involved—or at least infonned—in the planning stages of the OMS, then the 
groundwork would be laid for gaining their further support for new and revised policies that 
would improve treatment outcomes. 


Hypothetical Example: Culturally Specific Programs Found 
To Improve Outcomes 

A statewide analysis compared outcomes for patients of color served in special population programs 
with those served in general population programs. American Indian patients who were admitted to 
general programs were compared with those admitted to programs specially designed in the traditions 
of Native American cultures and staffed by American Indian staff. In this hypothetical analysis, no 







significant differences at admission were found between the two groups of American Indian patients. 
Although abstinence rates 6 months following treatment were comparable, patients in the American 
Indian programs reported better physical, psychological, social, and occupational functioning. The 
improvements were sufficient to justify a greater effort to make culturally distinct programming 
available to other communities of color. State funds were used for scholarships to promote minority 
counselor training and program development. 


Cultivating good relationships with both the governor's office and the legislature has always been 
an important part of the job performed by most SSAs. Many SSA staff members have been 
responsible for shepherding proposals of various types through the legislative process and are 
skilled in negotiating that process to achieve policy change. They understand the need to identify 
and take into account factors operating within the political environment, factors that can have a 
decisive effect on legislative outcomes. This kind of understanding and experience will be 
needed if outcomes monitoring is to serve its intended purpose. 

The State Legislature 

The support of legislators who are in a position to serve as advocates for policy regarding AOD 
treatment programs is imperative. This support should be cultivated throughout the process of 
developing and implementing the OMS. If possible, legislators or members of their staff should 
be represented on the steering committee that develops the OMS, although in some States 
regulations may prevent their formal participation. Even if legislative staff cannot formally 
participate on the committee, they should be invited to attend meetings as observers. 

In developing legislative support for recommendations resulting from outcomes monitoring, it 
will be helpful to identify a legislative champion. Who in the legislature cares about substance 
abuse issues and will provide leadership for gathering the support of colleagues? Such a 
champion or advocate may not necessarily be a lawmaker on an oversight committee, but 
someone with a more personal interest, perhaps a person recovering from a substance use 
disorder or the parent of a child who uses chemicals. 

It is likely that someone within the SSA has already established relationships with legislators 
who have this type of personal interest in supporting the agency and its mission. 

If no legislative advocate is readily identifiable, then the SSA must take steps to cultivate and 
develop such an advocate. The aid of an intermediary with close ties to a key legislator might be 
enlisted to make the initial contact with that legislator. 


Hypothetical Example: No Cost Benefits Found With New Vocational 
Training Treatment Component 

Suppose a statewide analysis were used to determine whether the addition of a vocational 
training component at several programs was worth the associated costs. The vocational 















component included training in job-seeking skills and practice in adhering to a structured work 
schedule. Patients who received this new service were compared with others with similar 
vocational deficiencies to determine whether there were differences in posttreatment abstinence, 
employment rates, job attendance, and job performance. In this hypothetical analysis, no benefits 
could be found either in terms of AOD use or vocational functioning. A decision was made to 
discontinue the new service and go back to the drawing board. 


Hypothetical Example: State Finds Its Services for Persons With 
Chronic Alcohol Dependence Inadequate 

Suppose a State with a fairly diverse array of services for persons with chronic alcohol 
dependence examined outcomes across a variety of programs throughout the State. None 
appeared to be especially effective in reducing alcohol use or its harmful health or social 
consequences. State agency personnel decided to explore options offered in other States to 
determine whether any other program model was producing better outcomes. Several States with 
similar data elements in their outcome monitoring systems collaborated on analyses to compare 
outcomes for patients with comparably severe profiles of alcohol use chronicity. 


Such an intermediary might be a legislator's spouse with an interest in substance abuse issues, or 
someone in the State office of the National Council on Alcoholism or some other appropriate 
constituency group. In California, for example, the Women’s Commission on Alcohol and Drug 
Dependence is a potent advocacy group with close links to the legislature and a good choice as 
an intermediary for the SSA. 

Once relationships with legislators and key committee staff have been established, it is essential 
to keep them informed. Techniques to accomplish this include regular briefings, interim reports, 
and presentation of an overview of projected findings and recommendations in a preliminary 
report. An informal presentation by the SSA director and several key staff members might be an 
effective way to make this report. The report might be given 3 months ahead of the scheduled 
publication of a final document. 

Such a presentation should be carefully planned, prepared, and polished. If the presentation 
includes information that the legislators or others in attendance may not want to hear, SSA staff 
should prepare them for that fact in advance. The presentation will be enhanced by effective 
audiovisuals that graphically depict the findings. It may also be enhanced by the participation of 
representatives of legislative constituencies, influential organizations, consumers, and families. 
There is some disagreement about the advisability of using consumer representatives to support 
agency positions and policies, and using them can carry some risk. But carefully selected 
consumers of services or family members, especially children, can provide powerful supporting 
testimony. 

Followup with all contacts with the legislature, as with other potential supporters and 
collaborators, should be timely and consistent. Once the formal report is published, press releases 















and other marketing efforts can encourage the legislative action necessary to implement needed 
policy changes. 

The Governor's Office 

Every effort should be made to ensure having the governor's support for findings and 
recommendations emerging from outcomes monitoring. In many States, clearance from the 
governor's office will be required before the legislature is contacted or any reports are issued. 

It is essential to the SSA's campaign for policy changes, that the SSA kn ow the established lines 
of authority and the appropriate channels of communication. It is important that the governor be 
represented on the steering committee and that someone in the governor's office be kept 
informed of all developments as they occur. 


Every effort should be made to ensure having the governor's support for findings and 
recommendations emerging from outcomes monitoring. 


There is probably already someone on the SSA staff who serves as liaison to the governor's 
office and who understands the process of effecting policy changes. This person may be the most 
appropriate choice to serve as liaison on behalf of outcomes monitoring, although the OMS 
project director may also assume this role. A key criterion for this assignment is the strength of 
the relationship with the governor's office. 

In some instances, the policy changes proposed by the SSA may be in conflict with the policies 
or political philosophy of the governor. When this is the case, managing the process of policy 
change becomes more complex, and the SSA may have to rely on constituency groups to lead 
lobbying efforts in support of desired changes. 

Treatment Providers and Local Officials 

Providers of substance abuse treatment services will be responsible for translating the results of 
outcomes monitoring into improvements in patient services. The SSA should communicate and 
interpret these results to providers in a supportive and nonthreatening way. Particularly in cases 
where program changes are called for, this task may require the same tact and diplomacy 
required to win the support of State policymakers. 

An OMS designed to address broad treatment service delivery questions will be less threatening 
to providers than one that compares individual program results. If OMS data indicate, for 
example, that outpatient counseling for certain patient groups does not work, then a different 
strategy must be tried. Programs offering outpatient counseling to these patient groups should 
know that the intent is not to eliminate them, but to find ways to make their efforts more 
effective. The emphasis is on improving patient outcomes, not casting blame or closing 
programs. 







At the same time it is working to reassure providers about the impact of OMS findings, the SSA 
must also be prepared to be directive to providers about programming in response to those 
findings and to assist them in making requisite changes. For example, OMS data might indicate 
that group counseling sessions are more effective than didactic education programs in improving 
patients' social functioning. A program already getting $50,000 to serve women with alcohol 
problems but that uses didactic strategies will need to revise its strategies based on such findings. 
Technical assistance from the SSA may be required, or it may be possible to refer the program 
director to another program serving similar patients but experiencing greater success through the 
use of group counseling. 

The single State agency's best approach to aiding and facilitating needed program revisions will 
vary from State to State, depending in part on the SSA's relationship with providers. Some 
agencies do not have statutory authority over providers. If a regional or county authority exists, it 
may be necessary to communicate findings through that authority. Existing communication 
channels may need to be strengthened in order to ensure appropriate feedback to programs. 
Regional meetings or other communication strategies in addition to dissemination of reports may 
be considered. 


Building Political Support for Policy Change 

• Monitor political forces within the State 

• Develop plans and strategies in collaboration with stakeholders 

• Find commonalities in positions 

• Point out ways in which SSA policy proposals are in the best interests of those whose 
support for their adoption is needed. 


Looking Forward 

It is important to bear in mind that outcomes monitoring should properly be considered an 
iterative process. Changes in outcomes variables, data collected, or analyses performed may be 
indicated following each data collection cycle. The steering committee may have a role in 
reviewing results and determining whether changes are needed in the OMS. The ongoing, 
evolving nature of outcomes monitoring must be understood by all participants in the OMS and 
all users of the resulting information. Similarly, the SSA should continue the process of 
education and cultivation of political allies. 

The cost of the OMS and the responsibility of the SSA will clearly go well beyond that 
associated with the data collection effort. The agency must ensure that data are appropriately 
analyzed and used and that findings are accepted. If done well, however, the benefits from 
outcomes monitoring should justify the effort and costs involved. 
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Appendix B—Outcomes Monitoring Systems 
of Some State Agencies 

Treatment outcomes monitoring systems developed by State agencies vary considerably in their 
content and methods as well as in their stage of development. Six State systems are profiled in 
this appendix to illustrate the variety as well as the commonalities of existing State systems: 
California, Colorado, Iowa, Minnesota, Tennessee, and Washington. 

California and Washington are included among the examples even though they do not conduct 
outcomes monitoring as described in this Treatment Improvement Protocol (TIP). These two 
States, like the other four discussed, have developed and implemented sophisticated systems to 
collect data on patients at admission and discharge. Like the others, they have incorporated the 
Federal Client Data Set in their admission forms and modified and enhanced this minimum data 
set in a variety of ways to meet local needs. Like the others, they also collect data on patients 
from a variety of levels of care and treatment modalities in their service delivery systems. 

California, with the largest treatment service delivery system among the States profiled, recently 
completed an expensive, methodologically rigorous followup study of a sample of patients to 
document treatment effectiveness and the cost benefits of treatment. This study involved in- 
person interviews that typically lasted over an hour, a method too expensive for broad-scale 
outcomes monitoring. While such a study is not a substitute for routine outcomes monitoring, it 
does dramatically illustrate the utility of a design that measures the cost offsets of treatment. 

Washington has not instituted postdischarge followup of its patients, but this is the next stage in 
its developmental process. This State system is included as an example of a State with a phased- 
in development and implementation plan. Also, Washington includes the most comprehensive 
assessment information on all patients, as well as a tracking mechanism for patients eligible for 
publicly funded treatment. Washington also has the most sophisticated system in terms of 
electronic data transfer and automated summary reports. The system is designed so that each 
program maintains its own individual database as well as forwards data for central processing. 

Among the States, only Minnesota collects data on virtually all private-pay as well as public-pay 
treatment patients. Minnesota's provider reimbursement system for public-pay clients is unique, 
resulting in a competitive marketplace where all programs in the State compete for public-pay 
clients. Since all but a few programs serve public-pay clients, all are subject to data collection 
requirements. In Iowa, private programs can report voluntarily, but few choose to do so. 

Of the four States described with systematic outcomes monitoring in place, Colorado, 

Minnesota, and Tennessee rely primarily on telephone interviews for patient followup. Iowa uses 
telephone and in-person interviews during clinic visits by the patient. Colorado and Iowa rely on 
treatment program staff to conduct the followup interviews, whereas Minnesota and Tennessee 
contract with researchers to conduct the interviews. Minnesota used treatment staff for several 
years but changed in 1993 because of the low contact rate and the skewed followup sample 
(composed largely of patients with better prognoses). Minnesota's contracted services include 



only the interviews; State agency staff conduct the data analyses and write the reports. 
Tennessee's contracted services include both data analyses and reports. 

Postdischarge followup intervals also vary from State to State. Iowa and Minnesota conduct a 
single followup at 6 months postdischarge. Tennessee conducts followups at 6 and 12 months 
and reports only on patients for whom both contacts were successful. In Colorado, the contact 
intervals range from approximately 12 to 18 months. The California outcome study also involved 
variable intervals ranging up to 24 months, with an average postdischarge interval of about 15 
months. 

Among the six States described here, only California has offered patients an incentive ($15) to 
participate in the followup interview. The interview was much longer (75 minutes) than those in 
other States, (about 15 to 20 minutes), and it was conducted in person rather than over the 
telephone. 

California, Colorado, Minnesota, and Tennessee all used or are using a sample of the total patient 
population for followup. California, Colorado, and Tennessee used a retrospective random 
sampling design, identifying the sample from discharge patients. Minnesota uses a prospective 
convenience sample. After all treatment program staff are trained, staff seek informed consent 
from 30 consecutive patients (or some predetermined pattern of alternate admissions). 

Consenting patients are asked to participate in additional comprehensive in-treatment data 
collection, as well as the followup interview. Iowa attempts to follow all patients who 
successfully complete treatment. 

Because of ongoing refinements to State systems, interested readers are encouraged to contact 
State agencies directly to request sets of questionnaires, forms completion manuals and other 
training materials, and available reports. The summary provided here also does not address 
followup consent rates and contact rates, both important to consider along with other 
methodological issues in weighing the potential effects of sample bias. Because these issues are 
too complex for the brief overview provided here, no attempt has been made to compare State 
systems in this regard. Outcomes monitoring system planners are urged to discuss design issues 
with State personnel with previous experience to determine why they chose the design they have 
adopted, what previous experiences influenced their decisions, and whether they have plans to 
make refinements in the future. 

California Department of Alcohol and Drug Programs 

Address: 1700 K Street 
Sacramento, CA 95814 

Contacts: Dennis Johnson 
Dorothy Torres 

Research and Policy Analysis Branch 
Telephone (916) 322-2285 
Fax(916)323-5873 


Treatment Data 


California Alcohol and Drug Data Systems (CADDS): 

Target programs. All treatment providers that receive any public funding for treatment services 
or that are required to report as a condition of State licensing; approximately 800 programs 
report on CADDS. 

Target population. All admissions to CADDS reporting programs; approximately 143,000 
annually. 

Levels of care. Three types of residential treatment (short-tenn, long-term, and residential) and 
four types of outpatient (methadone maintenance, nonmethadone, detoxification, and intensive). 
Forms. Participant Record Admission Copy, which incorporates the Federal Client Data Set. 
Participant Record Discharge Copy, which includes Client Data Set items at discharge as well as 
level of care, modality, and length of stay. 

Data entry. Keyboard. 

Followup Data 


California Drug and Alcohol Treatment Assessment (CALDATA): 

Followup method. Study design and interviews done under contract with the National Opinion 
Research Center (NORC) at the University of Chicago and Lewin-VHI, Inc., Fairfax, Virginia, 
between September 1992 and March 1994; in-person interviews with patient, lasting an average 
of 75 minutes, conducted by trained researchers from NORC. Average postdischarge interval 
was 15 months; intervals ranged up to 24 months. 

Target programs. Sample of 97 providers from 16 counties selected, based on principles of 
geographically balanced, size-weighted random selection. 

Target population. Random selection of patients discharged (or in methadone maintenance) 
between October 1, 1991, and September 30, 1992. A total of 1,859 interviews were completed 
from a base sample of 3,055. 

Data entry. Keyboard. 

Estimated costs. $2 million for followup study. 

Future Plans 

Conduct a 4- to 5-year followup on the patients interviewed for the 1994 report. 

Report 

Evaluating Recovery Services: The California Drug and Alcohol Treatment Assessment 
(CALDATA) General Report. Submitted to the State of California Department of Alcohol and 
Drug Programs by the National Opinion Research Center and Lewin-VHI, April 1994. Authored 
by Dean R. Gerstein, Robert A. Johnson, Henrick J. Harwood, Douglas Fountain, Natalie Suter, 
and Kathryn Malloy. A copy of the report can be obtained by contacting the California 
Department of Alcohol and Drug Programs Resource Center at (916) 327-3728 or (800) 879- 
2772. 



Colorado Department of Human Services Alcohol and Drug Abuse Division 
(ADAD) 

Address 4300 Cherry Creek Drive South 
Denver, CO 80222-1530 

Contact Linda Harrison 
Data Analysis and Evaluation Section 
Telephone (303) 692-2940 
Fax(303)753-9775 

Treatment Data 

Colorado Drug/Alcohol Coordinated Data System (DACODS): 

Target programs. All treatment providers that receive any public funding for treatment services, 
as well as those under special reporting requirements; approximately 40 programs. 

Target population. All admissions to reporting programs; approximately 63,000 annually. 
Levels of care. Free-standing residential detoxification, psychiatric residential, therapeutic 
community, transitional residential, intensive residential, and outpatient. 

Forms. DACODS Answer Sheet includes Admission-only items, items coded at Admission and 
Discharge, and Discharge- only items; incorporates Federal Client Data Set. 

Data entry. Keyboard; currently developing automated data submission whereby clinics would 
submit data on diskette. 

Followup Data 

ADAD Followup Study: 

Followup method. Telephone interviews with patient or third party conducted by treatment 
provider staff approximately 12 to 18 months postdischarge. 

Target programs. All programs except detoxification. 

Target population. Random selection from discharges between June and December 1992 
stratified by treatment history; redesign of study conducted about 2 years earlier. 

Data entry. Keyboard. 

Estimated costs. Personnel and other administrative costs. 

Future Plans 

Analyze and report on data collected at most recent followup. 

Publications 

Treatment Client Profiles and Followup Results, February 1992; Current Findings on Treatment 
Effectiveness, February 1992. 


Iowa Department of Public Health Division of Substance Abuse and Health 
Promotion 

Address Lucas State Office Building 
Des Moines, IA 50319 

Contacts Janet Zwick, Director 
Telephone (515) 281-4417 
Fax (515) 281-4958 

Linda Holt 

Telephone (515) 281-4643 
Fax (515) 281-4535 

Treatment Data 

Substance Abuse Reporting System (SARS): 

Target programs. All treatment providers that receive any public funding for treatment services, 
and private programs that voluntarily report. Approximately 46 programs report on SARS. 
Target population. All admissions to reporting programs; approximately 25,000 annually. 
Levels of care. Detoxification (medically managed, medically monitored, and outpatient), acute 
inpatient, medically monitored residential treatment, primary residential treatment, extended 
residential treatment, day treatment, halfway house, continuing care, extended outpatient, 
intensive outpatient. 

Forms. Admission/Evaluation fonn incorporates Federal Client Data Set. 

Services form records a variety of services and the number of days, sessions, or length of time 
involved in these services. 

Discharge/Followup form includes relevant Client Data Set items, discharge status, and 
discharge date, as well as outcome measures and ratings of services received. 

Data entry. More than half the programs enter data online and submit by modem or diskette; the 
others send hard copies for keyboard entry. 

Followup Data 

Uses SARS Discharge/Followup form: 

Followup method. Telephone interviews by treatment program staff or clinic visits 6 months 
postdischarge; followup conducted with patient or a significant other. 

Target programs. All reporting programs. 

Data entry. Same as above. 

Estimated costs. Personnel and other administrative costs. 


Minnesota Department of Human Services 

Chemical Dependency Division 

Address 444 Lafeyette Road 
St. Paul, MN 55155-3823 

Contact Patricia Ann Harrison, Ph.D. 

Administrative Planning Director 
Telephone (612) 296-8574 
Fax (612) 297-1862 

Treatment Data 

Drug and Alcohol Abuse Normative Evaluation System (DAANES): 

Target programs. All treatment providers that receive any public funding for treatment services 
(356 of 365 programs), including almost all private programs; approximately 35 detoxification 
facilities also report voluntarily on a separate DAANES system. 

Target population. All private- and public-pay admissions; approximately 32,000 treatment 
admissions and 36,000 detoxification admissions annually. 

Levels of care. Primary inpatient, primary outpatient, residential extended care, and halfway 
house; detoxification facilities report on a separate system. 

Forms. Intake form incorporates portion of Federal Client Data Set. 

History fonn incorporates an expanded version of the alcohol and other drug (AOD) use items 
from the Client Data Set and an arrest summary. 

Discharge form includes discharge status, referrals, payment source and charges, inpatient days, 
and outpatient treatment hours. 

Data entry. Optical scanner. 

Followup Data 

Treatment Accountability Plan (TAP): 

Followup method. Telephone interview with patient 6 months after discharge conducted by 
researchers at New Standards, Inc., St. Paul (formerly CATOR). 

Target programs. All reporting treatment programs (not detoxification facilities). Staff from 
approximately 15 programs are trained to begin TAP data collection each month for 24 months. 
Target population. 30 consecutive (or alternate) admissions from each program once during a 
3-year cycle. In addition to DAANES, TAP sample patients complete a modified Minnesota 
Addiction Severity Index (ASI) and weekly Treatment Services Review (TSR). 

Data entry. Online. 

Estimated costs. $20 per patient interview attempted or completed plus SSA personnel and 
administrative costs. 


Future Plans 


Complete followup interviews in 1995. Analyze data and issue final report on findings in 1996. Refine 
focus, revise instruments, and repeat the 3-year cycle. Develop software for onsite data entry of 
DAANES, ASI, and TSR and electronic data transfer by 1997. 

Publications 

Chemical Dependency Treatment Accountability Plan by Patricia Harrison. Report to the Minnesota 
Legislature, January 1992. The Minnesota Treatment Accountability Plan as a Treatment System 
Planning Tool by Patricia Harrison, submitted for publication 

Tennessee Department of Health 

Bureau of Alcohol and Drug Abuse Services 

Address Tennessee Tower 
312 8th Avenue North 
12th Floor 

Nashville, TN 37247-4401 

Contacts Rick Bradley 
Director of Contract Compliance 
Telephone (615) 741-8515 
Fax (615) 741-2491 

Charles Williams, Ph.D. 

University of Memphis Department of Anthropology 
Memphis, TN 38152 
Telephone (901) 678-2080 
Fax(901)678-2069 

Treatment Data 

Alcohol and Drug Services Admission form: 

Target programs. All treatment providers that receive any public funding for treatment 
services; approximately 55 agencies. 

Target population. All admissions to publicly funded programs; approximately 12,000 
annually. 

Levels of care. Residential and outpatient. 

Forms. Admission form incorporates Federal Client Data Set. 

Discharge form includes a record of services and number of visits, discharge status and referrals, 
rating of adaptive functioning at last encounter, and Client Data Set drug use items at 


termination. 

Data entry. Keyboard for half the agencies; the others submit data on diskette; converting to 
diskette data submission for all programs in 1995. 

Followup Data 

Tennessee Outcomes for Alcohol and Drug Services (TOADS): 

Followup method. Telephone interviews with patients and collaterals conducted by research 
assistants under supervision of project director at University of Memphis; interviews conducted 6 
and 12 months postdischarge (report sample limited to patients who complete both interviews). 
Target programs. 25 facilities in 1991/1992; expanded to all programs willing to participate. 
Target population. Random selection of patients who complete treatment; sample totaled 1,846 
in fiscal years 1991 and 1992. Future target of 15 percent of all admissions. 

Incentives. None. 

Data entry. Optical scanner. 

Estimated costs. Approximately $225,000 annually for followup interviews and reports. 

Future Plans 

Analyze and report on data collected since September of 1993. 

Publications 

A Report of Outcomes: Tennessee Outcomes for Alcohol and Drug Services (TOADS). Prepared for the 
Bureau of Alcohol and Drug Abuse Services, Tennessee Department of Health, by Charles Williams and 
Nancy Hepler, October 1994 

Washington State Department of Social and Health Services, Division of 
Alcohol and Substance Abuse (DASA) 

Address P.O. Box 45330 
Olympia, WA 98504-5330 

Contacts Kenneth Stark, Director 
Telephone (206) 438-8200 
Fax (206) 438-8078 

Fritz Wrede 

Telephone (206) 438-8224 
Fax (206) 438-8078 

Treatment Data 


Treatment and Assessment Report Generation Tool (TARGET): 


Target programs. All treatment providers that receive any public funding for treatment 
services; approximately 194 programs. 

Target population. All admissions to publicly funded programs; approximately 13,000 
detoxification admissions and 32,000 treatment admissions annually. 

Levels of care. Detoxification, intensive inpatient, intensive outpatient, outpatient methadone, 
long-term residential, extended care, recovery house. 

Forms. Assessment/Admission Setup form includes patient identification and demographics, 
provider identification, and referral information for patients eligible for publicly funded 
treatment. 

Assessment/Admission and Discharge form includes Federal Client Data Set items as well as 
assessment information related to physical health, psychological condition, illegal activity, and 
substance use history. 

Discharge form includes discharge status, referrals. 

Data entry. Online; each participating agency maintains its own local database as well as 
forwarding data to central database. System features a variety of automated reports. 

Followup Data 

Tentative plans include telephone followup conducted by an independent research agency under 
contract with DASA; followup sample would be drawn from TARGET database and findings integrated 
with TARGET data. Plans also include a series of special studies on special populations and areas of 
particular interest. 



Appendix C—Federal Resource Panel 

Brad Austin, M.P.H. 

Public Health Advisor 

Treatment Operations and Review Branch 

Division of State Programs 

Center for Substance Abuse Treatment 

Substance Abuse and Mental Health Services Administration 

Dorynne Czechowicz, M.D. 

Associate Director for Medical and Professional Affairs 
Medical Affairs Branch 
Division of Clinical Research 
National Institute on Drug Abuse 

Herman Diesenhaus, Ph.D. 

Review Administrator 

Office of Scientific Analysis and Evaluation 

Center for Substance Abuse Treatment 

Substance Abuse and Mental Health Services Administration 

Bennett W. Fletcher, Ph.D. 

Research Psychologist 
Treatment Services Research Branch 
Division of Clinical Research 
National Institute on Drug Abuse 

Anita Gadzuk 

Public Health Analyst 
Office of Applied Studies 

Substance Abuse and Mental Health Services Administration 

Patricia Ann Harrison, Ph.D. - Chair 

Administrative Planning Director 
Chemical Dependency Program Division 
Minnesota Department of Human Services 

Linda Kaplan, M.A., C.A.E. 

Executive Director 

National Association of Alcoholism and Drug Abuse Counselors 

Anna Marsh, Ph.D. 

Associate Director for Evaluation 
Office of Applied Studies 



Center for Substance Abuse Treatment 

Substance Abuse and Mental Health Services Administration 

Eleanor M. Perfetto, Ph.D., R.Ph. 

Senior Phannacoepidemiologist 

Center for Medical Effectiveness Research 

Agency for Health Care Policy and Research 

Barbara Ray, Ph.D. 

Senior Policy Analyst 

Office of Applied Studies 

Center for Substance Abuse Treatment 

Substance Abuse and Mental Health Services Administration 

Richard T. Suchinsky, M.D. 

Associate Director for Addictive Disorders and Psychiatric Rehabilitation 
U.S. Department of Veterans Affairs 

Betty Tai, Ph.D. 

Director, Clinical Cocaine Program 
Medications Development Division 
National Institute on Drug Abuse 

Nancy K. Young, Ph.D. 

Academic Coordinator/Research Consultant 

School of Social Welfare for the California Department of Alcohol and 
Drug Programs 

University of California, Los Angeles 



Appendix D—Field Reviewers 

Thomas F. Babor, Ph.D., M.P.H. 

Professor and Scientific Director 
Alcohol Research Center 
Department of Psychiatry 
University of Connecticut 
Health Center 
Fannington, Connecticut 

C. B. Bautista, M.S. 

Deputy Director 

Planning and Public Policy Division 
Department of Alcohol and Drug Programs 
Sacramento, California 

Edward Bernstein, M.D. 

Associate Professor 

Emergency Medicine and Public Health 
Boston City Hospital 
Boston, Massachusetts 

Jack Beveridge 

Executive Director 

Pinal Gila Behavioral Health Association 
Apache Junction, Arizona 

Bill B. Burnett, M.Ed., N.C.A.C. II 

Director of Intervention and Treatment 

South Carolina Department of Alcohol and Other Drug Abuse Services 
Columbia, South Carolina 

Juesta M. Caddell, Ph.D. 

Research Clinical Psychologist 

Research Triangle Institute and Duke University Medical Center 
Research Triangle Park, North Carolina 

Dorynne Czechowicz, M.D. 

Associate Director for Medical and Professional Affairs 
Division of Clinical Research 
National Institute on Drug Abuse 

Herman I. Diesenhaus, Ph.D. 

Review Administrator 

Office of Scientific Analysis and Evaluation 



Center for Substance Abuse Treatment 
Substance Abuse and Mental Health Services 
Administration 

Lee Dixon 

Deputy Director 

Intergovernmental Health Policy Project 
George Washington University 
Washington, D.C. 

Chris M. Edwards, M.S.W., A.C.S.W. 

Clinical Director 
SMI Services 

Arizona Center for Clinical Management 
Tucson, Arizona 

John A. Fairbank, Ph.D. 

Senior Research Clinical Psychologist 
Research Triangle Institute 
Research Triangle Park, North Carolina 

James Fidelibus, Ph.D. 

Director of Research and Development 
Administrative Offices 

INTERACT Behavioral Healthcare Services, Inc. 
Columbus, Ohio 

Gary Field, Ph.D. 

Alcohol and Drug Program Manager 
Oregon Department of Corrections 
Salem, Oregon 

Pamela Ann Finton, M.A. 

Director of Quality Assurance 
Amity Teaching Communities 
Tucson, Arizona 

Robert Fiorentine, Ph.D. 

Principal Investigator 
Drug Abuse Research Center 
UCLA Neuropsychiatric Institute 
University of California 
Los Angeles, California 

Deborah Ford, R.N. 

Service Center Manager 



Treatment and Rehabilitation for Addicts in Need II 
Washington, D.C. 

William E. Ford, Ph.D. 

Vice President and Director 
Integrated Behavioral Services 
Mutual of Omaha Companies 
Mutual of Omaha Plaza 
Omaha, Nebraska 

Anita Gadzuk 

Public Health Analyst 

Office of Applied Studies 

Substance Abuse and Mental Health Services 

Administration 

Susan Galbraith, M.S.W. 

Co-Director of National Policy 
Legal Action Center 
Washington, D.C. 

Fred Garcia, M.S.W. 

Deputy Director 

Office of Demand Reduction 

Office of National Drug Control Policy 

Executive Office of the President 

Jan Johnson Grest 

President 

DataLock, Inc. 

Arden Hills, Minnesota 
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Exhibit 5-1 

Patient and Treatment Variables 


Patient Variables 


Administrative 

Predictors 

Baseline/Outcome 

Patient identifier 

Demographics 

AOD use frequency 

Referral sources 

Education 

AOD use amount 

Payment source 

Vocational history 

Mode of drug administration 


Social history 

HIV risk behaviors 


Alcohol and other drug (AOD) 
use history 

AOD treatment history 

Medical history 

Psychiatric history 

Legal problems 

Motivation 

Treatment readiness 

AOD dependence symptoms 

Physical health 

Psychological health 

Employment 

Financial stability 

Legal problems 

Family/social relationships 

Treatment Variables 

Administrative 

Predictors 


Program identifier 

Setting/level of care 


Admission date 

Therapeutic modality 





























Discharge date 

Context 


Length of stay 

Treatment components 


Charges 

Staffing 

Discharge status 

Patient satisfaction 

Posttreatment services 



No single patient assessment instrument will ever achieve universal acceptance; the AOD field is 
too diverse for that. The ASI, like any instrument, has its limitations and its critics. Some of the 
limitations and ways to compensate for them are discussed in the sections that follow. While 
consensus panel members were reluctant to recommend any one instrument as the standard for 
State outcomes monitoring systems, it was proposed that the ASI or its constituent variables be 
given serious consideration because of the strong base of research supporting its validity, 
reliability, and utility. Furthennore, its content addresses most of the areas recommended for 
consideration. 


Patient Variables 

As explained above, patient variables can be historical one-time measures (predictor variables), 
or they can measure status prior to treatment at different points along the treatment continuum or 
following treatment (baseline/outcome measures). For purposes of an OMS, data collected at 
intake or shortly thereafter should include both historical and baseline measures. 


While consensus panel members were reluctant to recommend any one instrument as the standard 
for State outcomes monitoring systems, it was proposed that the ASI or its constituent variables be 
given serious consideration because of the strong base of research supporting its validity, reliability, 
and utility. Furthermore, its content addresses most of the areas recommended for consideration. 


Patient predictor variables include characteristics generally referred to as demographics such as 
gender, age, and race/ethnicity, as well as social/vocational factors such as marital status, 
employment status, level of education, and income level. Predictor variables indicate special 
treatment needs. These variables also include aspects of a patient’s history that may include such 



















factors as a history of physical or sexual abuse, childhood conduct disorder or antisocial 
behavior, age of onset of substance use, number of previous treatments and admissions for 
detoxification, psychiatric hospitalizations, chronic medical conditions, and many other factors. 
These will be discussed in detail later in this chapter. 


Exhibit 5-2 

Federal Client Minimum Data Set (1990) 

Required Items 

Optional Items 

PROVIDER IDENTIFIER 

PRIMARY AOD PROBLEM 

MARITAL STATUS 

PATIENT IDENTIFIER 

Alcohol 

Never married 


Cocaine/crack 

Now married 

DATE OF ADMISSION 

Marijuana/hashish 

Separated 


Heroin 

Divorced 

DATE OF BIRTH 

Methadone 

(nonprescription) 

Widowed 

SEX 

Other opiates and 
synthetics 

LIVING ARRANGEMENTS 

Male 

PCP 

Homeless 

Female 

Other hallucinogens 

Dependent living 


Methamphetamines 

Independent living 

RACE 

Other amphetamines 



Other stimulants 

PRIMARY SOURCE OF 

Alaska Native 

Benzodiazepines 

INCOME/SUPPORT 

American Indian 

Other tranquilizers 


Asian or Pacific 

Barbiturates 

Wages/salary 

Islander 

Other sedatives or 

Public assistance 

Black 

hypnotics 

Retirement/pension 

White 

Inhalants 

Disability 

Other 

Over-the-counter 

drugs 

Other 

ETHNICITY 

Other 

None 

HEALTH INSURANCE 

Puerto Rican 

SECONDARY AOD PROBLEM 


Mexican 

(choices same as 

None 

Cuban 

Medicare 

Other Hispanic 

primary) 

Medicaid 

Not of Hispanic Origin 

TERTIARY AOD 

Private insurance 















HIGHEST SCHOOL GRADE 
COMPLETED 


PROBLEM 


EMPLOYMENT STATUS 

Employed full time 
Employed part time 
Unemployed 
Not in the labor force 

(Optional 

breakdowns) 

Homemaker 

Student 

Retired 

Disabled 

Inmate of 

institution 

PRINCIPAL REFERRAL 
SOURCE 

An individual (or self) 

AOD abuse care 

provider 

Other health care 

provider 

School 

Employer/EAP 
Other community 
referral 

Court/criminal justice 
referral 

(Optional CJ referral 
categories) 

State/Federal 

court 

Other formal 
adjudication 
process 

Probation/parole 

Other 


(choices same as 
primary) 

AGE OF FIRST USE (OR 

ALCOHOL 

INTOXICATION) 

USUAL ROUTE OF 
ADMINISTRATION 

Oral 

Smoking 

Inhalation 

Injection 

Other 

FREQUENCY OF AOD USE 

No use past month 
1-3 times past month 
1-2 times per week 
3-6 times per week 
Daily 

LEVEL OF SERVICE 

Hospital inpatient 

detoxification 

Free-standing 

residential 

detoxification 

Hospital rehabilitation 

(acute care) 

Short-term (<30 days) 
nonacute residential 
Long-term (>30 days) 
nonacute residential 
Intensive outpatient (2+ 
hours per day 3+ 
days per week) 
Nonintensive outpatient 
Outpatient 


Blue Cross/Blue Shield 

Health maintenance 

organization 

Other 

Unknown 

EXPECTED PRIMARY SOURCE 
OF PAYMENT FOR THIS 
TREATMENT EPISODE 

Self-pay 

Workers compensation 

Medicare 

Medicaid 

Other government 
payments 

Blue Cross/Blue Shield 

Other health insurance 

No charge 

Other 

Unknown 

DSM-lll-R DIAGNOSTIC CODES 
PSYCHIATRIC PROBLEM 

Yes 

No 

PREGNANT AT ADMISSION 

Yes 

No 

VETERAN STATUS 

Yes 

No 

DAYS WAITING TO ENTER 
TREATMENT 











recognized legal 

detoxification 


entity 



DUI/DWI 

METHADONE PLANNED AS PART 


OF TREATMENT 


Diversionary 


program 

Yes 


Prison 

No 


PRIOR AOD TREATMENT 

INITIAL ADMISSION IN 


ADMISSIONS 

TREATMENT EPISODE VERSUS 


0 

1 

2 

TRANSFER IN SERVICE 

PATIENT VERSUS CODEPENDENT 


OR COLLATERAL RECORD 


3 

4 

5 or more 




The distinction between predictor variables and baseline/outcome measures is more an issue of 
form than substance. Some variables can fit both categories, depending on when they are 
measured and whether the question is repeated at treatment followup. For example, the number 
of lifetime detoxification admissions can be collected at intake simply as an indicator of chronic 
intoxication. But admissions to detoxification could also be used as a baseline/outcome measure 
if compared over a defined interval before and after treatment (for example, the number of 
detoxification admissions during the 6 months before treatment compared with the number 
during the 6 months after treatment). Similarly, a psychiatric diagnostic variable could be used as 
a predictor variable, while measures of recent psychological distress could be used as 
baseline/outcome measures. 

Administrative Patient Variables 

Patient Identifier 

A patient identifier is critical to linking various forms for the same patient and for linking 
followup questionnaires to treatment questionnaires. It can also be used to match the records of 
multiple admissions for the same patient. 

There are basically two types of patient identifiers—those that can be traced back to an 
individual and those that cannot. Examples of identifiers that can be traced to an individual 
include name, social security number, driver's license number, and permanent numbers assigned 
to recipients of public assistance. The concern about using traceable identifiers in an OMS is that 
their use is a breach of patient confidentiality. Identifiers that cannot be traced to an individual 
include cryptograms comprised typically of some combination of the following: letters from the 











first, middle, and/or last name; date of birth; digits from the social security number; or other 
identifying elements that do not change over time. The drawback of nontraceable identifiers is 
that they cannot be used to match records with other information systems that do not use the 
same type of identifiers (such as Medicaid, child protection, arrest records, and the like). 
Whichever type of patient identifier is used, extreme caution must be taken to protect patient 
confidentiality. All patient identifiers can be "scrambled" upon data entry so they are 
decipherable only to a few designated individuals who know the code. Computer databases can 
also be protected by several layers of security. The legal issues related to protecting 
confidentiality are discussed in Chapter 6. 


Whichever type of patient identifier is used, extreme caution must be taken to protect patient 
confidentiality. 


Referral Sources 

The Federal Client Data Set includes a required item for principal referral source. One alternative 
would be to expand the response choices (for instance, separating self and family and adding 
more specific choices under community and healthcare referrals) and then collapse them for 
Federal reporting purposes. Another alternative would be to have the option "check all that 
apply" in addition to a "principal" referral. The intended use of the data should guide the final 
decision. 

Payment Source 

An optional item in the Federal Client Data Set lists response categories for "Expected primary 
source of payment for this treatment episode." In most States, it would probably be more useful 
to identify the actual payment source or sources. Since this item may not be detennined at intake, 
it may more practical to include this question on a discharge form. 

Patient Predictor Variables 

Demographics 

Demographic information such as age, gender, and race/ethnicity has obvious descriptive 
importance in an outcomes monitoring system. To the extent that differences in outcomes might 
be observed among major demographic groups, this information must be taken into account in 
planning services. 

These variables serve a number of useful functions. Aggregated across programs at a State level, 
they provide a descriptive profile of who is entering treatment. They can provide comprehensive 
profiles of patients being served by the treatment system or any portion of it. By comparing this 
information with population base rates and AOD incidence/prevalence rates, it is also possible to 
use these data to determine who is not being served by AOD treatment programs. 







The patient demographic profile can also be used to track trends in treatment admissions over 
time. Changing patterns might suggest the need for more programs designed to serve special 
populations. For example, if an increasing number of women are entering treatment, this increase 
may indicate a need for more programs to serve this population. Similarly, if a decreasing 
number of adolescents are receiving AOD treatment, the capacity for this group may need to be 
decreased. 


The patient demographic profile can also be used to track trends in treatment admissions over time. 
Changing patterns might suggest the need for more programs designed to serve special populations. 


Demographic information can also be cross-tabulated or correlated with other variables such as 
AOD use patterns and treatment discharge status to detennine whether demographic factors are 
independent predictors of outcomes or interact with other variables to predict outcomes. 
Controlling for demographic factors in data analyses can also indicate when demographic 
predictor variables are merely artifacts of other predictor variables. 

Frequently in social science research, a demographic factor is significantly associated with an 
outcome. For example, race may be significantly associated with likelihood of relapse. Flowever, 
race may also be significantly associated with a primary drug problem. For instance, the rate of 
treatment admissions for cocaine addiction may be higher among blacks than whites in the 
geographic area under study. An analysis that controlled for drug of choice might find that once 
drug of choice was controlled for, there was no significant difference between the outcomes of 
blacks and whites; that is, whites and blacks with alcohol problems had comparable outcomes, 
and whites and blacks with cocaine problems had comparable outcomes. 

Typically, the kinds of variables included under demographics are the easiest to turn into specific 
questions on data collection instruments. They are usually formatted as categorical variables, 
where one response is selected from among a list of responses, and thus are much easier to 
design than measures of problem severity. Nonetheless, some important considerations should 
not be overlooked. These will be explained in the following discussion when pertinent. 

Gender and age. Gender is an essential variable and one of the few that needs no discussion. 
Age is also extremely important as a predictor of outcomes and relatively straightforward to 
collect. In some systems that include birth date and date of admission, age is computed rather 
than recorded. 

Race/ethnicity. Many instruments include the race breakdown used in the 1990 U.S. Census and 
the Federal Client Data Set. Five racial categories are used (Alaska Native, American Indian, 
Asian or Pacific Islander, Black, and White) with an extra response choice for "Other"; a 
separate variable addresses Hispanic ethnicity. Hispanic people can be of any race. While these 
formats have advantages in terms of standardization, they can pose some problems. Biracial and 
multiracial patients are sometimes categorized into one race by the interviewer based on 







appearance; sometimes patients are asked to pick which racial category best describes them; and 
sometimes biracial and multiracial patients are categorized as "other." 

An alternative to the forced choice option is a "Check all that apply" response format. Several 
choices can then be formatted into single categories for Federal reporting purposes, but the value 
of more accurate and detailed descriptions would not be lost. 

Another potential problem with existing standard racial/ethnic categories is that they may not 
provide sufficient information for some localities. For example, agencies in States with relatively 
large American Indian populations may wish to include tribal affiliation as a subset of the Indian 
category. States with diverse Asian American populations may find it helpful to distinguish 
Southeast Asian and other Asian subgroups. For example, forms used in California and 
Washington State include more than 10 specific Asian designations. 

On the other hand, some persons argue against collecting data on race because results can be 
misinterpreted or used for political ends. Findings attributed to race actually often reflect cultural 
or socioeconomic factors. It is important to remember that even widely used variables and 
response choices should be examined in terms of whether they will provide information in ways 
most appropriate for local purposes. 

Primary language/immigration status. Language difference is another variable that may help 
State planners design programs that are appropriate to the populations they serve. The 
Washington State system includes a question asking whether use of English is functional or 
limited, as well as an item identifying the primary functional language from among a list of 46 
languages, including American Sign Language. Recent immigration status may also be important 
for cultural considerations and special needs, particularly in those States with large numbers of 
immigrants. 


Some persons argue against collecting data on race because results can be misinterpreted or used for 
political ends. Findings attributed to race actually often reflect cultural or socioeconomic factors. 


Education 

The Federal Client Data Set uses the highest school grade completed as a measure of education 
level (a general equivalency diploma is equivalent to 12 years). Another option is a categorical 
variable that includes as response choices the highest degree attained: none, high school diploma, 
vocational technical certificate, bachelor's degree, master's degree, or professional degree (e.g., 
M.D., J.D., or other doctoral degrees). 

Social History 

Marital/relationship status, parenthood, sexual orientation. Marital status is an optional item 
on the Federal Client Data Set. The item as it exists does not provide a category for persons 







living in a committed relationship but not legally married. OMS planners might want to consider 
different response choices for this existing item or perhaps add an additional item. 

Information on parenthood status and number of dependents can also help with program design 
and may be predictive of outcomes. The Washington State forms include questions about the 
number of children living with the patient (their own and others'), as well as children not living 
with the patient. 

Sexual orientation can be a controversial item, and there may be doubt about the truthfulness of 
patients' responses because of their fears of discrimination. Some proponents argue that it is 
important to determine whether gay, lesbian, bisexual, or transgender individuals are more likely 
than others to have AOD problems. It is important to remember, of course, that treatment 
admissions are not an accurate reflection of AOD abuse prevalence. If underreporting were to 
occur, it would distort the findings anyway. As with all other variables under consideration, the 
key question is utility. Is the question asked routinely as part of individual treatment planning? 
Would the system be changed in any way as a result of findings in this area? If the data are not 
going to be put to practical use, it is probably not justifiable to ask patients to reveal such 
sensitive infonnation. 

Living arrangement. One simple breakdown is provided as an optional item in the Federal 
Client Data Set: homeless or transient, dependent living (as with parents or in a supervised 
setting), or independent living. These latter two response choices could be broken down into 
subcategories. For instance, dependent living could include separate response choices for living 
with parents, living with other relatives, board and lodging, halfway house, and so forth. 
Independent living could include separate choices for house, apartment, or mobile home. The 
utility of more detailed information should guide the decision process. 

Veteran status. Veteran status is also an optional item in the Federal Client Data Set. The item 
is easy to collect and noncontroversial but may have limited utility overall. Minnesota modified 
the item so the "Yes" response was changed to "Yes, no combat" and "Yes, served in combat 
zone" to see whether combat experience might be a predictor variable. Iowa has modified this 
question to address military status, with response choices of "none," "veteran," "in reserves," and 
"active duty." 

Socioeconomic status, income, health insurance. Socioeconomic status (SES) is a useful 
concept but not necessarily an easy one to operationalize. OMS planners may want to review 
available SES classifications to detennine whether any would be useful for their purposes. 
Several optional items from the Federal Client Data Set can be used as rough SES indicators, for 
example, primary source of income/support, health insurance, and expected source of payment 
for treatment. Other options include personal or family income and occupational level. The State 
of Washington asks for monthly household income and monthly personal income. Iowa asks for 
taxable individual monthly income as well as occupation level (professional/managerial, 
sales/clerical, crafts/operatives, nonfarm laborers, farm owners and laborers, and 
service/household). 



Health insurance variables can be useful since health insurance plays a part in dictating the kind 
of healthcare to which a person has access, which may have a significant relationship to 
outcomes. While any one variable in this general area has its limitations, a selection of several of 
these variables will probably provide sufficient information for categorizing patients. 

Spirituality/religious affiliation. Another factor that can be considered as part of social history 
is religious affiliation and spirituality. Recovering AOD abusers often emphasize the importance 
of spirituality in their recovery. Involvement in church activities is another predictor of positive 
outcomes. Many AOD treatment programs use the 12-step approach, and issues of religiosity and 
spirituality may be important and can be measured (Tonigan et al., 1991). 

Family and Other Social Relationships 

The ASI addresses several aspects of family and social relationships. As lifetime measures, it 
determines whether close personal relationships existed with parents, siblings, partner(s), 
children, and friends. It also asks about periods of serious problems with these people as well as 
with other family members, neighbors, and coworkers. (This section, like all others in the ASI, 
also includes many 30-day measures for baseline/outcome comparison.) 

Other variables could certainly be used. The key question, in terms of both treatment planning 
and predictive value, would be the availability and extent of social supports. 

History of physical/sexual abuse. Many adolescents and adults in treatment report histories of 
sexual abuse and family violence. It is not yet known to what extent these factors may be 
predictors of outcome. Documentation of the extent of these problems among AOD patients, 
however, can be used for treatment planning and design purposes. 

Many instruments, including the ASI, require a yes or no response to questions about abuse, 
victimization, and/or perpetration. For example, the Washington questionnaire asks whether the 
patient has ever been a victim of domestic violence and whether the patient is currently a victim 
of domestic violence. While this format allows for quantification of the extent of these problems, 
it may be inadequate as a predictor of treatment outcome. Many factors contribute to the impact 
of physical and sexual abuse on its victims: identity of the perpetrator, age of the victim, 
frequency and severity of abuse, and whether the victim sought or received help at the time or 
since the abuse. Assessment of all these issues, while appropriate for clinical purposes, would be 
beyond the scope of an OMS. 

Abuse victimization and perpetration can also be used as baseline/outcome measures and will be 
discussed in this context later in this chapter. 

Vocational History 

Employment status. A variable that can present problems is employment status. The Federal 
Client Data Set includes a forced-choice response format: employed full time, employed part 
time, unemployed (and looking for work), or not in the labor force. This last category can be 
expanded into another forced-choice format that includes homemaker, student, disabled, retired, 


and correctional inmate. A forced choice results in the loss of information for patients who have 
multiple roles and responsibilities, such as the homemaker who is employed and goes to school, 
or the student who is also employed. Multiple role status in itself may be a predictor of outcomes 
and thus valuable information to retain. Forced choices by their very nature reflect a value as to 
which is more important when multiple responses apply, a value which may not be shared by 
patients or other OMS stakeholders. In the case of the Federal Client Data Set, "employed full 
time" would supersede choices listed below, and other applicable information would be lost. 

Loss of information can be avoided with a "check all that apply" format. For Federal reporting 
purposes, the responses can be recoded into the hierarchy of response choices set forth in the 
Client Data Set guidelines. While this TIP cannot address the nuances of all variable options in 
this detail, the employment status variable, like the race variable, points out the pitfalls of 
assuming that demographics are easy variables to define or agree on. 

Employment history. Current employment status does not provide information about 
occupational stability over time or give any indication of downward mobility that might be 
associated with AOD use. The ASI includes variables on length of longest held full-time job, 
most recent or current occupational status, and usual employment pattern over the previous 3 
years. The Iowa system inquires about the number of months worked during the previous 6 
months. These are examples of variables that inquire beyond employment status at admission. 


Current employment status does not provide information about occupational stability over time or 
give any indication of downward mobility that might be associated with AOD use. 


AOD Use History 

Information about the history of substance use is important for proper evaluation of treatment 
outcomes because the type of drug and the severity of the AOD problem may dictate the 
intensity and specificity of treatment as well as predict outcome. People with a substance use 
disorder are a heterogeneous population with a variety of different use patterns; however, the 
effects of all these differences on outcomes are unclear. 

Because reduction in substance use and its associated consequences is one of the primary goals 
of treatment, many substance use variables are appropriate as baseline/outcome measures and 
will be discussed later in this chapter. However, some substance use variables can be used as 
predictors as well, and some are solely one-time measures. 

Substances used, primary drugs of abuse. Some indication of the substances used by 
individual patients is essential to an OMS. There are various ways to collect this information, 
and several combine type of drug (or drug class) with use frequency. The Federal Client Data Set 
requires coding the drug (or drugs) identified as primary problem, secondary problem, and 
tertiary problem for each patient. Clinical judgment is used to make the determination. For 
primary and secondary drugs, codes are also required for age of onset of use (first intoxication 







for alcohol), frequency of use, and usual mode of administration. This data set has some 
limitations, which will be discussed below. 

Age of onset. Age of onset of use has been shown to be a predictor of substance use severity in 
both epidemiological and clinical studies (Buydens-Branchey et ah, 1989). In the Federal Client 
Data Set, this variable is required only for primary and secondary drugs of abuse. If the patient 
initiated use with a third drug, the required information does not provide actual age of onset of 
any AOD use. This limitation can be resolved by either asking for this directly or including age 
of onset for all drug or drug categories reported by the patient. 

Frequency of use. In the Federal Client Data Set, this variable is also required only for primary 
and secondary drugs of abuse. If the patient uses a variety of drugs, no information is obtained 
on the remainder. The variable is further limited by focusing only on recent use. A solution to 
both these problems is to add frequency of use as a variable for all drugs reported. It may be 
preferable, however, to exercise this option as a baseline/outcome measure so that frequency of 
use over a specified time interval can be compared before and after treatment. Use frequency will 
be discussed in greater detail in the section on baseline/outcome measures. 

Mode of drug administration. "Usual mode of administration" is included in the Federal Client 
Data Set. Because it is required only for primary and secondary drugs of abuse, it does not serve 
to identify all current (or previous) injection drug users. It only identifies those clients whose 
usual mode of administration is injection. This information may be deemed inadequate, 
considering the risks for HIV infection associated with even infrequent use, or previous use 
dating back many years. Several States, including California, Iowa, Minnesota, and Washington, 
have included a more general question to address any history of injection drug use. This question 
need not be drug specific. 


Age of onset of use has been shown to be a predictor of substance use severity in both 
epidemiological and clinical studies. 


AOD diagnoses. Substance use disorder diagnoses recorded at intake or discharge may be useful 
for purposes of population profiles. The Diagnostic and Statistical Manual of Mental Disorders, 
Third Edition, Revised (DSM-III-R) (American Psychiatric Association, 1987) codes are optional 
variables in the Federal Client Data Set. However, diagnostic categories alone are of limited 
value as predictor variables and should be supplemented by other indicators of problem severity 
such as frequency of use and use consequences. The full series of AOD abuse/dependence 
diagnostic codes should be listed with instructions to "check all that apply" in order to capture 
the full range of applicable diagnoses for poly drug-using patients. 

Treatment History 

Treatment history is an important patient variable to measure. It can provide an efficient 
indicator of the severity of a patient's alcohol or drug problem. The number of lifetime treatment 







admissions and number of lifetime detoxification admissions, for example, can assist with 
treatment placement decisions and may be useful predictors of outcome. The number of prior 
AOD treatment admissions is included in the Federal Client Data Set. Iowa limits the period of 
inquiry to 10 years. 

Medical History 

In many studies, a relationship has been found between AOD use and injuries, physical 
disabilities, and other medical problems (Cherpitel, 1992; Cregler, 1989; Eckardt et ah, 1981; 
Heinemann, 1993; Moessner, 1979). Whether to collect information on medical history and 
current medical conditions and the amount of detail to include will depend on the purposes of the 
OMS and the availability of resources. Background information could include such data as 
number of lifetime hospitalizations (excluding childbirth) or a general rating scale of overall 
physical health. On the other hand, if more detail is sought, questions could be asked about the 
nature and duration of illnesses or symptom severity. 

While it is true that many illnesses are associated with substance abuse, the purpose of collecting 
exhaustive medical data in an OMS must be justified. What might be appropriate for a medical 
workup is not necessarily appropriate for an outcomes monitoring system. This is another area 
that could generate an almost unlimited number of potential variables, and decisions must be 
guided by the utility of the information in analyzing treatment outcomes. If the purpose is to 
determine whether ill health is related to outcomes, general indicators will probably suffice. If 
the purpose is to determine to what extent medical problems remit with treatment, a 
baseline/outcome measure would be needed. 


The purpose of collecting exhaustive medical data in an OMS must be justified. What might be 
appropriate for a medical workup is not necessarily appropriate for an outcomes monitoring system. 
Decisions must be guided by the utility of the information in analyzing treatment outcomes. 


The ASI includes questions about lifetime medical hospitalizations, chronic medical problems, 
and regular prescription drug use. The Washington system includes questions covering numbers 
of emergency room visits, outpatient/clinic visits, inpatient hospitalizations, and days of 
hospitalization in the previous year. Washington also has separate questions that address whether 
or not the patient is currently under medical care for infectious disease, traumatic injury, 
continuing illness, or dental problems. 

Pregnancy status. Pregnancy status is an optional and controversial item in the Federal Client 
Data Set. In some States, legislation has been passed that mandates reporting certain illicit drug 
use by pregnant women to local authorities. This requirement may motivate women to deny 
pregnancy. Depending on whether this item was completed at intake or shortly thereafter, 
pregnancy may also be underreported. Some women may discover they are pregnant at a later 
point during treatment (or not know at all). 







Psychiatric History 


Psychiatric disorders are found in AOD-abusing populations at rates higher than in general 
populations (Helzer and Pryzbeck, 1988; Hovens et ah, 1994; Regier et ah, 1990; Rounsaville, 
1990; Stowell and Estroff, 1992). Psychiatric disorders may predate AOD abuse, be a 
consequence of AOD abuse, or coexist with AOD abuse. Psychiatric disorders include 
depression and bipolar disorder (manic-depression), anxiety disorders, schizophrenia and other 
thought disorders, posttraumatic stress disorder, dissociative disorders, and eating disorders and 
other compulsive behavior disorders. They also include personality disorders such as antisocial 
personality disorder. (For more information about dual diagnosis of psychiatric disorders and 
AOD abuse, refer to another TIP in this series, Assessment and Treatment Planning for Patients 
With Coexisting Mental Illness and Alcohol and Other Drug Abuse?) 

As with physical disorders, it is important to be selective about the amount of information 
collected about psychiatric disorders and emotional distress. Collecting and recording data for an 
OMS is not the same as clinical assessment or documentation. Again, the guiding principle in 
OMS planning is to select those variables that may be significantly associated with treatment 
outcome or that serve as baseline/outcome measures. For example, the amount of subjective 
distress experienced by a patient may have a greater relationship to outcome than to the specific 
nature of the psychiatric diagnosis, or a combination of the diagnostic code and clinical severity 
rating may provide the most useful information. Ultimately, for purposes of data analysis, large 
numbers of variables must be distilled into a smaller number of useful ones. A review of existing 
studies and outcome monitoring systems may guide the distillation process. Incorporating a 
variety of psychiatric assessment instruments into the OMS may seem like a safe way to be sure 
all the bases are covered, but it is also likely to pose an insurmountable obstacle in terms of staff 
and patient time and data analysis. 


The amount of subjective distress experienced by a patient may have a greater relationship to 
outcome than to the specific nature of the psychiatric diagnosis, or a combination of the diagnostic 
code and clinical severity rating may provide the most useful information. 


The Federal Client Data Set includes an optional variable that merely records the existence of a 
psychiatric problem. This question is probably too general to be of any predictive value and too 
vague to provide useful interpretation of findings. Washington includes an item to record 
whether or not a psychological evaluation was conducted and a psychiatric diagnosis was made, 
as well as yes/no questions on the current use of mental health services and psychotropic 
medication. 

The ASI includes questions on the number of lifetime psychiatric hospitalizations and courses of 
outpatient treatment. The ASI also asks about lifetime episodes of depression, anxiety, 
hallucinations, memory or concentration difficulties, difficulty controlling violent behavior, 
suicidal thoughts, and suicide attempts, as well as the use of psychotropic medication. The 
Minnesota OMS added a question to the ASI series to address other compulsive behaviors such 







as eating disorders and gambling. Similarly, Iowa asks a general question about addictions other 
than AOD dependence. 

Motivation to Enter Treatment, Treatment Readiness, and Coercive Influences 

Patients enter treatment for many reasons. Factors that influence the decision may be internal or 
external, and for many patients a variety of factors may impinge on the decision. For some, the 
admission is involuntary; they are court ordered or committed to treatment. For others, coercion 
may be more subtle: the threat of a spouse's or partner's leaving or the loss of a job, a 
professional license, or custody of children are often significant factors. Patients may also enter 
treatment because they are sick, broke, or weary of maintaining their addiction. 

An OMS may include a measure of the coercive influences and motivations to enter treatment in 
order to determine their relationship to treatment outcomes. Findings in this area could have 
important policy implications and could be used to improve patient outcomes. A rating of patient 
readiness for treatment could also be included. If patient readiness were found to be a strong 
predict or of positive outcomes, it might indicate a change in patient placement strategies or the 
need to search for ways to improve patient readiness. 

In Minnesota, the OMS includes a question that addresses the primary condition surrounding 
admission to treatment. Response choices include treatment versus incarceration or as a 
condition of probation or parole, to avoid loss of children or to regain their custody, to prevent 
loss of a relationship or living situation, to regain a driver's license, to keep a professional license 
or job or to stay in school, to retain eligibility for government benefits, or to escape financial 
pressures related to continued alcohol and drug abuse. 

Baseline/Outcome Variables 

The key to valid and meaningful baseline/outcome measures is a comparable interval of 
measurement before and after treatment; Chapter 4 addresses this issue in detail. The advantages 
of short windows, such as 30 days, are more accurate recall and finer precision of measurement. 
A drawback is the lack of information for a longer duration. In fact, this lack is one of the 
criticisms of the ASI, which measures AOD use frequency only for the previous 30 days. These 
questions, asked 6 months after discharge from treatment, would not reveal whether the patient 
had used any alcohol or other drugs since treatment. In contrast, a 6-month interval covers a 
longer duration but lacks precision in terms of measurement. The recommended solution is a 
combination of both short-term and long-term intervals. This combination will be explained 
more in the discussion of specific variables that follows. 

Frequency of AOD Use 

The most obvious baseline and outcome measures are those associated with substance use and 
abuse before and after treatment. Measures of frequency of AOD use are almost always included 
in treatment outcome studies. Frequency of use is important to measure because it is associated 
with a range of biopsycho-social problems. Reduction of use frequency is a useful outcome 
measure even in the absence of total abstinence. A number of instruments incorporate effective 


ways to ask about frequency of AOD use. The ASI includes use frequency for a variety of drugs 
and drug categories but limits the inquiry to the previous 30 days. 


Measures of frequency of AOD use are almost always included in treatment outcome studies. 
Frequency of use is important to measure because it is associated with a range of biopsychosocial 
problems. 


Minnesota's OMS design addressed the limitations of both the Federal Client Data Set and the 
ASI in measuring AOD use frequency. For all the Client Data Set drug categories, use frequency 
responses are: 

• No use in past month 

• 1-3 times in past month 

• 1-2 times per week 

• 3-6 times per week 

• Daily. 

The Minnesota OMS replaces the "no use in past month" response choice with three other 
choices: 

• Used in past 2-6 months (but not in past month) 

• Used in lifetime (but not in past 6 months) 

• Never used. 

This design captures the information as required for the CDS but also allows for a record of 6- 
month and lifetime use. (The lifetime response choice is used only at admission, not at 
followup.) 

AOD Use Amounts 

The amount of drug use per typical occasion of use can be an indicator of the severity of the 
AOD problem and the individual's ability to maintain control over use. Amount of use is not 
used as often as frequency of use because it is more difficult to quantify and typically less valid 
and reliable. For example, while quantity of alcohol consumed during a typical drinking occasion 
may be a useful measure for social drinkers, its value diminishes for persons who abuse or are 
dependent on alcohol. Alcoholics may drink from the bottle, so a measure of number of "drinks" 
becomes meaningless. Even when persons drink cans of beer, glasses of wine, or mixed drinks, 
their accuracy in reporting quantity is likely to decline with increased consumption (Webb et ah, 
1991). 

Amount of use of illicit drugs is even more difficult to quantify. There is no standard measure of 
drug use comparable to that for alcohol. Potency is not controlled by governmental regulation. 
Amount of consumption (number of joints, hits, lines, or fixes) may provide some indication of 








the compulsiveness of the drug-taking behavior, but it does not indicate how much of the drug 
was actually ingested. Drug quantity is not recommended as an OMS measure. 


The amount of drug use per typical occasion of use can be an indicator of the severity of the AOD 
problem and the individual's ability to maintain control over use. 


Mode of Drug Administration 

Mode of drug administration is an important variable for a number of reasons. Injection drug use 
is an indication of problem severity and also important in terms of the risk of transmission of 
HIV and other infectious diseases. The Client Data Set includes "usual mode of administration" 
at admission and the same questions could be asked at followup. Another option is to specifically 
ask for the frequency of injection (the mode of administration of greatest concern) for an 
identical time interval before and after treatment. 

HIV Risk Behaviors 

Injection drug use has already been addressed. In Minnesota, another question addresses the 
frequency of using "clean works" among patients who report injection drug use. High-risk sexual 
behaviors have also been associated with AOD use, either because of poor judgment under the 
influence or the use of sex as a commodity to obtain drugs. Questions can address the use of 
condoms during sexual intercourse and the number of sexual partners over a specified interval 
before and after treatment. 

These are obviously highly sensitive topics. In Minnesota, where questions about using clean 
works are included in the OMS, these questions are the ones most likely to draw objections from 
former patients during followup telephone interviews. Nonetheless, they may prove useful 
measures of behavior change. 

AOD Use Dependence Symptoms 

Measures of symptoms of dependence are useful in determining the severity of a relapse 
following treatment. Dependence symptoms include: 

• Loss of control over use 

• Unsuccessful attempts to control use 

• Tolerance 

• Withdrawal symptoms. 

According to the fourth edition of DSM (American Psychiatric Association, 1994), a remission 
has been achieved when abuse and dependence symptoms have been absent at least 1 month. 
Early remission covers the first 12 months without symptoms, and sustained remission covers the 
period beyond 12 months. Remission can be further categorized as partial or full. In full 







remission, no abuse or dependence symptoms are present; in partial remission, full criteria for 
dependence are not met, but at least one criterion is present either intermittently or continuously. 
Including DSM-IV criteria symptoms in the OMS will allow for classification of patients' 
posttreatment status in terms of partial or full remission (or partial or full recovery). 

The Federal Client Data Set includes diagnostic codes from the third revised edition of DSM as 
optional items. (In DSM-IV, the latest edition of DSM, the diagnostic codes for abuse of and 
dependence on particular substances are the same as in the third revised edition.) However, 
diagnosis is likely to be established in a clinical setting as a routine part of the assessment 
process. Including all the relevant questions to establish diagnosis at followup may be too time 
consuming, relative to the benefits of this particular element. The Minnesota OMS limits inquiry 
in this area to a question on the number of times the patient experienced withdrawal symptoms in 
a 6-month period before and after treatment. 

Physical Health Measures 

AOD use can have direct and indirect effects on physical health. Direct effects include, for 
example, infections from injection drug use and damage to the liver and other organs from 
alcohol. Indirect effects result from poor nutrition, unsanitary living conditions, lack of sleep, 
and other health risks associated with AOD use and AOD-related lifestyles, such as traumatic 
injury. 

It is sufficient for purposes of an OMS to measure health status before and after treatment, 
without necessarily linking health status to AOD use. For example, the ASI asks, "How many 
days have you experienced medical problems in the past 30 days?" as well as "How troubled or 
bothered have you been by these medical problems in the past 30 days?" and "How important to 
you now is treatment for these medical problems?" Reduction in problem days or subjective 
severity rating can be used to show improved status after treatment. 


It is sufficient for purposes of an OMS to measure health status before and after treatment, without 
necessarily linking health status to AOD use. 


Other options are questions that record the number of days in the hospital or visits for emergency 
care during, for example, a 6-month period before and after treatment. 

General medical care may actually increase for a time during or after treatment as patients attend 
to long-neglected health issues. For this reason, measures of outpatient physician or clinic visits 
do not always show a posttreatment reduction. Inpatient and emergency care, in contrast, 
typically show dramatic decreases (California Department of Alcohol and Drug Programs, 1994; 
Harrison and Hoffmann, 1989). 







Psychological Health Measures 


As with physical health, AOD use can have direct and indirect consequences for psychiatric 
status and emotional well-being. AOD use can exacerbate existing mental health problems or 
bring on depression and difficulties with memory or concentration. Living conditions and 
disrupted interpersonal relationships associated with severe addiction can also cause emotional 
distress. Finally, difficulty controlling violent behavior, suicidal thoughts, and suicide attempts 
can also be associated with AOD use. 

The ASI asks about a variety of psychological symptoms over the previous 30 days, information 
which provides useful baseline/outcome measures. The psychological questions parallel the 
medical questions regarding number of days (in the past 30) on which emotional problems were 
experienced, a subjective rating of problem severity, and a rating of the need for treatment. 

Psychiatric hospitalization over a 6-month period or longer could be used as a baseline outcome 
measure; however, base rates are relatively low, limiting the utility of this item. Outpatient visits 
may actually increase as patients follow through on treatment plans that include attention to 
mental health as well as AOD use. 

Employment Measures 

Current employment status, a Federal Client Data Set item, can be used at admission and at the 
posttreatment contact. Months of employment in the preceding 6 months is another option. 

The ASI uses a 30-day measurement window and several variables: days paid for working, net 
employment income, days experiencing employment problems, subjective rating of employment 
problem severity, and rating of the need for counseling for problems related to employment. 

Financial Stability Measures 

Relying solely on employment variables to assess financial stability is inadequate because a large 
number of patients may not be in the work force (such as students, homemakers, retired, and 
disabled persons). Measures of financial stability can be used to indicate change in status before 
and after treatment. The ASI includes 30-day income measures for unemployment compensation, 
public assistance, and pensions or social security. 

The Minnesota modification of the ASI incorporates questions about financial stability 
comparable to the employment questions: days experiencing financial problems, subjective 
rating of financial problem severity, and rating of the need for assistance with financial 
problems. 

Legal Problem Measures 

Legal problems can be quantified by using measures such as number of arrests over a specified 
interval before and after treatment. Researchers with experience in this area often prefer arrests 
to actual crimes because arrests are a matter of public record, making it more likely that patients 


will give an honest accounting. A recent California study was very successful at quantifying 
actual crimes (California Department of Alcohol and Drug Programs, 1994), but the typical 
OMS is not likely to have comparable resources available. 


Legal problems can be quantified by using measures such as number of arrests over a specified 
interval before and after treatment. 


The ASI uses two 30-day measures: number of days detained or incarcerated and number of days 
engaged in illegal activities for profit. It also includes the patient's subjective rating of severity of 
legal problems and need for counseling or referral for those problems. 

Family/Social Relationships 

Marital or relationship status can be used at baseline and at outcome, but this information can be 
used to indicate whether the status is an improvement. Living arrangement may also be of some 
limited utility. It may be more helpful to measure satisfaction with marital/relationship status and 
living situation. 

The ASI includes 30-day questions about interpersonal conflicts that provide outcome measures, 
but the absence of conflict may mean merely lack of social contact rather than improved 
relationships. The ASI also includes 30-day measures of sexual, physical, and emotional abuse— 
whether or not the patient has been abused or has abused anyone else. 

The Minnesota modification of the ASI adds 30-day measures regarding family stressors other 
than conflict. It also includes a rating of how troubled the patient has been by loneliness or being 
alone, as well as questions about living with people who use alcohol or other drugs or who have 
AOD problems. 

The limitation of these instruments is that neither the ASI nor Minnesota's modification 
addresses changes in positive aspects of family or social relationships, especially the 
establishment of new relationships by patients who had previously been isolated or estranged 
from others. 

Treatment Variables 

Much remains u nkn own about how outcomes relate to the type and quantity of treatment 
received, and this relationship is an issue of concern to the single State agencies. In outcomes 
research, treatment variables have typically been neglected (Sobell et ah, 1987), but they have 
important implications for understanding outcomes. Certainly they have important cost 
ramifications, as in the areas of setting and staffing. If different treatment modalities or programs 
can deliver comparable quality of care as measured by comparable outcomes, then cost alone 
might be the basis for choosing one over another. 







If only baselines and outcomes are considered, then treatment itself is the "black box," with what 
goes on within treatment remaining a mystery to outcomes evaluators and policymakers. For an 
OMS to function effectively, there must be a level of specificity about what goes on between 
patient admission and discharge. In the past, for example, researchers have been preoccupied 
with comparing the results of outpatient versus inpatient programs (Cummings, 1991; Miller and 
Hester, 1986), without looking at the actual content of these programs. Issues that must be 
considered include differences in forms of treatment, such as individual, group, and family 
therapy; differences in settings and populations; differences in lengths of stay and frequency and 
intensity of treatment sessions; and differences in the actual components of treatment. 


If only baselines and outcomes are considered, then treatment itself is the "black box," with what 
goes on within treatment remaining a mystery to outcomes evaluators and policymakers. 


Administrative Treatment Variables 

Program Identifier 

An identifier is necessary to distinguish one treatment provider from another. This is a 
noncontroversial item. 

Admission Date, Discharge Date, and Length Of Stay 

For inpatient and residential treatment programs, length of stay can be computed if the admission 
date is recorded on an intake form and the discharge date on a discharge form. For outpatient 
programs, however, this information only indicates length of time from start to finish of 
treatment and gives no indication of number of contacts during that interval. Hours of 
involvement or contact days could be recorded on a discharge form. Service amounts will be 
discussed in more detail later. 

Level of Service 

Level of service is a required item in the Federal Client Data Set. This breakdown or an 
expansion of it should be included in an OMS. The CDS also requires an indication of whether 
an admission is the initial admission in a treatment episode or a continuation of treatment 
resulting from a transfer (such as inpatient to outpatient care). Level of care can also be 
considered a predictor variable. 

Treatment Charges 

Information on costs associated with treatment is essential to an OMS focused on accountability 
for taxpayer dollars. Information could be collected as a package cost for the whole program or 
as itemized costs for specific services. Costs are not readily available to the program staff who 







complete patient forms, however. As some States move to managed care based on capitated 
rates, costs for individual treatment admissions may be difficult or impossible to compute. 


As some States move to managed care based on capitated rates, costs for individual treatment 
admissions may be difficult or impossible to compute. 


Treatment Predictor Variables 

Therapeutic Modality and Social Climate 

Little work has been done in determining the effect of therapeutic focus on patient outcomes for 
different kinds of patients. For instance, while therapeutic communities differ from 12-step 
programs, there is still not enough in formation available to predict which patients would do best 
in which modality. 

Social climate may also affect patient outcomes. For example, a well- kn own program may be 
extremely similar in structure and focus to many other programs, but its high profile may lead 
patients to have greater expectations of success. 

Treatment Components 

While a measure of therapeutic focus might be important, it might be more revealing to include a 
variety of measures of the specific services that comprise the treatment regimen. Most programs 
actually consist of a composite of some of the following components: 

• Individual, group, or family counseling sessions 

• Medical and psychiatric/psychological evaluation and treatment 

• Educational lectures, films, or reading material 

• Dietary and nutritional guidance 

• Education, referral, counseling, testing, and treatment for HIV/AIDS, tuberculosis, and other 
infectious diseases 

• Educational and vocational services 

• Social services including housing, childcare, and transportation 

• Legal assistance 

• Recreational and occupational therapy 

• Independent skills training 

• Biofeedback 

• Acupuncture 

• Psychotropic medication 

• Opioid substitution therapy 

• Exercise 

• Spiritual counseling 

• Others. 








Some treatment components may be specific to cultural heritage and traditional healing 
practices. 

Decisions must be made not only about which treatment components should be measured, but 
also how to measure them. While instruments that attempt to define and quantify treatment 
components are relative newcomers in the field compared with those that assess patient 
characteristics, they should be examined for their usefulness in this area. For instance, a simple 
checklist could be used to indicate whether or not a patient had received a service, while a more 
sophisticated measure could be developed to describe the frequency, intensity, and duration of 
the service. It would be important to know, for example, whether or not a certain minimum 
amount of a service were necessary to achieve any beneficial effect, or conversely, whether or 
not a threshold were reached beyond which there were little or no measurable benefit. While no 
OMS could measure all the potential treatment variables that could be generated, attention to 
selected key variables may produce useful in formation for allocating limited resources, placing 
patients in the most appropriate settings, and giving them the services most likely to produce 
favorable outcomes. 


It would be important to know whether a certain minimum amount of a treatment service were 
necessary to achieve any beneficial effect, or conversely, whether a threshold were reached beyond 
which there were little or no measurable benefit to the patient. 


The Treatment Services Review (McLellan et ah, 1992a) provides a standard measure of the 
nature and number of services provided during the treatment process. Services are categorized 
along the same dimensions covered by the ASI and are recorded weekly. For the Minnesota 
OMS, the Treatment Services Record—a similar instrument—was developed (Harrison, in 
press). This instrument also uses the ASI dimensions to categorize services and quantifies hours 
in treatment and in specific components of treatment (educational sessions; individual, group, 
and family counseling sessions; support group meetings; and informal contact with other 
patients). Both the Treatment Services Review and the Treatment Services Record measure 
contacts that are part of formal treatment as well as ancillary services, since both may affect 
outcomes. The Iowa OMS also quantifies time involved in specific services, recorded on a 
monthly basis. 

Staffing Patterns 

Staffing patterns can be described by considering several objective and subjective qualities of 
staff members. These include the following: 

• Professional training and availability of staff with specialized training to do specialty work 

• Recovery status and prior AOD involvement of counseling staff 

• Therapist characteristics such as empathetic versus confrontational, directive versus facilitative 

• Management style 

• Staff morale 








• Amount of staff contact with patient. This contact may vary considerably in both duration and 
intensity. 

Few data exist on the relation of staffing patterns to patient outcomes. Selectivity in measuring 
the staff qualities listed above would be imperative if staff qualities were to be included as 
measures in an OMS. Most patients are likely to have contact with a variety of staff, so they may 
be exposed to a variety of levels of expertise, AOD use histories, and therapist characteristics. In 
addition, staff characteristics may not be static and may interact in different ways with individual 
patient characteristics. Although staffing characteristics and staffing patterns may be related to 
treatment outcomes, a decision might be made not to include these measures in an OMS because 
of lack of data about the benefits of collecting this information in relation to the time and costs 
involved. 

Context/Special Populations Served 

Programs vary tremendously in terms of the populations they serve. Special populations can be 
defined by personal characteristics such as gender, age, race/ethnicity, cultural background, or 
sexual orientation. Because of Federal block grant mandates, many States are currently designing 
more programs for pregnant women and women with children. Special populations can also be 
defined by drug use patterns; some programs are designed specifically for people addicted to 
heroin or cocaine. Programs can also be designed for individuals with coexisting disorders such 
as mental illness, eating disorders, or compulsive gambling. If an OMS were to find that certain 
patients did better in programs designed to serve their special needs, compared with other 
settings, these results could be used to justify increased funding for such targeted services, 
despite the increased costs these might entail. 

Discharge Status 

The patient's treatment completion status is essential information to collect in an OMS. 
Successful treatment completion can be used as an intermediary outcome measure (Wickizer et 
ah, 1994), whereas leaving against staff advice may predict a poorer posttreatment outcome. 
Other reasons for failure to complete treatment may provide useful information to improve 
treatment retention. 


The patient's treatment completion status is essential information to collect in an OMS. Successful 
treatment completion can be used as an intermediary outcome measure, whereas leaving against 
staff advice may predict a poorer posttreatment outcome. 


Patient Satisfaction 

Patient satisfaction with treatment as a whole or with specific services received can be measured 
during and after treatment. Satisfaction ratings by service type are included in the Minnesota 
weekly Treatment Services Record as well as in the followup interview. The Iowa system also 







includes patient ratings of the benefits of certain treatment components collected at followup. 
While patient ratings of treatment may not be related to other treatment outcomes, they may be 
important to consider in an era that is becoming more attuned to customer satisfaction. 
Satisfaction may also be related to treatment retention, so a correlation between patient 
dissatisfaction and early dropout from treatment may provide useful information about program 
changes necessary to keep patients in treatment. 

Posttreatment Services 

The use of community resources following discharge from treatment may also have an impact on 
patient outcomes. Patients are typically referred to peer support groups and other available 
resources to assist with their recovery following discharge from formal treatment. Learning more 
about the relation of these services and successful treatment outcomes can assist with discharge 
planning and resource allocation. 


Existing State Systems 

The content included in existing State outcomes monitoring systems provides a basis for review 
by States planning a new system or revising their current systems. Several State systems, in 
varying stages of development, are described in Appendix B. A contact resource is listed for 
each, to aid readers in obtaining a set of current instruments and other relevant information. 
Because of their length and because revisions occur periodically, individual forms are not 
reproduced in this TIP. 

All the discussion of OMS content in this chapter is based on experience to date. In this rapidly 
evolving field, recommendations will change as new information emerges from existing systems 
and ongoing studies. Some of the data elements currently in widespread use may be found to be 
of limited utility in evaluating outcomes. Others, in trial stages or not yet developed, may prove 
to offer more benefits. While this TIP can provide a useful overview of outcomes monitoring 
systems and general guidelines for content, decisions for an individual State will depend on local 
needs and an up-to-date review of continuing improvements in this field. 


Exhibit 5-2 

Federal Client Minimum Data Set (1990) 

Required Items 

Optional Items 

PROVIDER IDENTIFIER 

PRIMARY AOD PROBLEM 

MARITAL STATUS 

PATIENT IDENTIFIER 

Alcohol 

Never married 


Cocaine/crack 

Now married 

DATE OF ADMISSION 

Marijuana/hashish 

Separated 


Heroin 

Divorced 

DATE OF BIRTH 

Methadone 

(nonprescription) 

Widowed 

SEX 

Other opiates and 
synthetics 

LIVING ARRANGEMENTS 

Male 

PCP 

Homeless 

Female 

Other hallucinogens 

Dependent living 


Methamphetamines 

Independent living 

RACE 

Other amphetamines 



Other stimulants 

PRIMARY SOURCE OF 

Alaska Native 

Benzodiazepines 

INCOME/SUPPORT 

American Indian 

Other tranquilizers 


Asian or Pacific 

Barbiturates 

Wages/salary 

Islander 

Other sedatives or 

Public assistance 

Black 

hypnotics 

Retirement/pension 

White 

Inhalants 

Disability 

Other 

Over-the-counter 

drugs 

Other 

ETHNICITY 

Other 

None 

HEALTH INSURANCE 

Puerto Rican 

SECONDARY AOD PROBLEM 


Mexican 

(choices same as 

None 

Cuban 

Medicare 

Other Hispanic 

primary) 

Medicaid 

Not of Hispanic Origin 

TERTIARY AOD 

Private insurance 

HIGHEST SCHOOL GRADE 

PROBLEM 

Blue Cross/Blue Shield 

COMPLETED 


Health maintenance 


(choices same as 

organization 

EMPLOYMENT STATUS 

primary) 

Other 

Unknown 

Employed full time 

AGE OF FIRST USE (OR 


Employed part time 

ALCOHOL 

EXPECTED PRIMARY SOURCE 















Unemployed 

INTOXICATION) 

OF PAYMENT FOR THIS 

Not in the labor force 

USUAL ROUTE OF 

TREATMENT EPISODE 

(Optional 

ADMINISTRATION 

Self-pay 

breakdowns) 

Oral 

Workers compensation 

Homemaker 

Smoking 

Medicare 

Student 

Inhalation 

Medicaid 

Retired 

Injection 

Other government 

Disabled 

Other 

payments 

Inmate of 


Blue Cross/Blue Shield 

institution 

FREQUENCY OF AOD USE 

Other health insurance 


No use past month 

No charge 

PRINCIPAL REFERRAL 

1-3 times past month 

Other 

SOURCE 

1-2 times per week 

Unknown 

An individual (or self) 

3-6 times per week 

DSM-lll-R DIAGNOSTIC CODES 

AOD abuse care 

Daily 

PSYCHIATRIC PROBLEM 

provider 

LEVEL OF SERVICE 

Yes 

Other health care 

provider 

Hospital inpatient 

No 

School 

detoxification 


Employer/E AP 

Other community 

Free-standing 

PREGNANT AT ADMISSION 

referral 

residential 

Yes 

Court/criminal justice 

detoxification 

No 

referral 

Hospital rehabilitation 


(Optional CJ referral 

(acute care) 

VETERAN STATUS 

categories) 

Short-term (<30 days) 

Yes 

State/Federal 

nonacute residential 

Long-term (>30 days) 

No 

court 

nonacute residential 

DAYS WAITING TO ENTER 

Other formal 

Intensive outpatient (2+ 


TREATMENT 

adjudication 

hours per day 3+ 


process 

days per week) 


Probation/parole 

Nonintensive outpatient 


Other 

Outpatient 

detoxification 


recognized legal 


entity 



DUI/DWI 

METHADONE PLANNED AS PART 


Diversionary 

OF TREATMENT 


program 



Prison 

Yes 












PRIOR AOD TREATMENT 

No 


ADMISSIONS 

INITIAL ADMISSION IN 


0 

TREATMENT EPISODE VERSUS 


1 

TRANSFER IN SERVICE 


2 

PATIENT VERSUS CODEPENDENT 


3 

4 

5 or more 

OR COLLATERAL RECORD 



The distinction between predictor variables and baseline/outcome measures is more an issue of 
form than substance. Some variables can fit both categories, depending on when they are 
measured and whether the question is repeated at treatment followup. For example, the number 
of lifetime detoxification admissions can be collected at intake simply as an indicator of chronic 
intoxication. But admissions to detoxification could also be used as a baseline/outcome measure 
if compared over a defined interval before and after treatment (for example, the number of 
detoxification admissions during the 6 months before treatment compared with the number 
during the 6 months after treatment). Similarly, a psychiatric diagnostic variable could be used as 
a predictor variable, while measures of recent psychological distress could be used as 
baseline/outcome measures. 

Administrative Patient Variables 

Patient Identifier 

A patient identifier is critical to linking various forms for the same patient and for linking 
followup questionnaires to treatment questionnaires. It can also be used to match the records of 
multiple admissions for the same patient. 

There are basically two types of patient identifiers—those that can be traced back to an 
individual and those that cannot. Examples of identifiers that can be traced to an individual 
include name, social security number, driver's license number, and permanent numbers assigned 
to recipients of public assistance. The concern about using traceable identifiers in an OMS is that 
their use is a breach of patient confidentiality. Identifiers that cannot be traced to an individual 
include cryptograms comprised typically of some combination of the following: letters from the 
first, middle, and/or last name; date of birth; digits from the social security number; or other 
identifying elements that do not change over time. The drawback of nontraceable identifiers is 
that they cannot be used to match records with other infonnation systems that do not use the 
same type of identifiers (such as Medicaid, child protection, arrest records, and the like). 
Whichever type of patient identifier is used, extreme caution must be taken to protect patient 
confidentiality. All patient identifiers can be "scrambled" upon data entry so they are 
decipherable only to a few designated individuals who know the code. Computer databases can 











also be protected by several layers of security. The legal issues related to protecting 
confidentiality are discussed in Chapter 6. 


Whichever type of patient identifier is used, extreme caution must be taken to protect patient 
confidentiality. 


Referral Sources 

The Federal Client Data Set includes a required item for principal referral source. One alternative 
would be to expand the response choices (for instance, separating self and family and adding 
more specific choices under community and healthcare referrals) and then collapse them for 
Federal reporting purposes. Another alternative would be to have the option "check all that 
apply" in addition to a "principal" referral. The intended use of the data should guide the final 
decision. 

Payment Source 

An optional item in the Federal Client Data Set lists response categories for "Expected primary 
source of payment for this treatment episode." In most States, it would probably be more useful 
to identify the actual payment source or sources. Since this item may not be determined at intake, 
it may more practical to include this question on a discharge form. 

Patient Predictor Variables 

Demographics 

Demographic information such as age, gender, and race/ethnicity has obvious descriptive 
importance in an outcomes monitoring system. To the extent that differences in outcomes might 
be observed among major demographic groups, this information must be taken into account in 
planning services. 

These variables serve a number of useful functions. Aggregated across programs at a State level, 
they provide a descriptive profile of who is entering treatment. They can provide comprehensive 
profiles of patients being served by the treatment system or any portion of it. By comparing this 
in formation with population base rates and AOD incidence/prevalence rates, it is also possible to 
use these data to detennine who is not being served by AOD treatment programs. 

The patient demographic profile can also be used to track trends in treatment admissions over 
time. Changing patterns might suggest the need for more programs designed to serve special 
populations. For example, if an increasing number of women are entering treatment, this increase 
may indicate a need for more programs to serve this population. Similarly, if a decreasing 
number of adolescents are receiving AOD treatment, the capacity for this group may need to be 
decreased. 







The patient demographic profile can also be used to track trends in treatment admissions over time. 
Changing patterns might suggest the need for more programs designed to serve special populations. 


Demographic information can also be cross-tabulated or correlated with other variables such as 
AOD use patterns and treatment discharge status to detennine whether demographic factors are 
independent predictors of outcomes or interact with other variables to predict outcomes. 
Controlling for demographic factors in data analyses can also indicate when demographic 
predictor variables are merely artifacts of other predictor variables. 

Frequently in social science research, a demographic factor is significantly associated with an 
outcome. For example, race may be significantly associated with likelihood of relapse. Flowever, 
race may also be significantly associated with a primary drug problem. For instance, the rate of 
treatment admissions for cocaine addiction may be higher among blacks than whites in the 
geographic area under study. An analysis that controlled for drug of choice might find that once 
drug of choice was controlled for, there was no significant difference between the outcomes of 
blacks and whites; that is, whites and blacks with alcohol problems had comparable outcomes, 
and whites and blacks with cocaine problems had comparable outcomes. 

Typically, the kinds of variables included under demographics are the easiest to turn into specific 
questions on data collection instruments. They are usually formatted as categorical variables, 
where one response is selected from among a list of responses, and thus are much easier to 
design than measures of problem severity. Nonetheless, some important considerations should 
not be overlooked. These will be explained in the following discussion when pertinent. 

Gender and age. Gender is an essential variable and one of the few that needs no discussion. 
Age is also extremely important as a predictor of outcomes and relatively straightforward to 
collect. In some systems that include birth date and date of admission, age is computed rather 
than recorded. 

Race/ethnicity. Many instruments include the race breakdown used in the 1990 U.S. Census and 
the Federal Client Data Set. Five racial categories are used (Alaska Native, American Indian, 
Asian or Pacific Islander, Black, and White) with an extra response choice for "Other"; a 
separate variable addresses Hispanic ethnicity. Hispanic people can be of any race. While these 
formats have advantages in terms of standardization, they can pose some problems. Biracial and 
multiracial patients are sometimes categorized into one race by the interviewer based on 
appearance; sometimes patients are asked to pick which racial category best describes them; and 
sometimes biracial and multiracial patients are categorized as "other." 

An alternative to the forced choice option is a "Check all that apply" response format. Several 
choices can then be formatted into single categories for Federal reporting purposes, but the value 
of more accurate and detailed descriptions would not be lost. 







Another potential problem with existing standard racial/ethnic categories is that they may not 
provide sufficient information for some localities. For example, agencies in States with relatively 
large American Indian populations may wish to include tribal affiliation as a subset of the Indian 
category. States with diverse Asian American populations may find it helpful to distinguish 
Southeast Asian and other Asian subgroups. For example, forms used in California and 
Washington State include more than 10 specific Asian designations. 

On the other hand, some persons argue against collecting data on race because results can be 
misinterpreted or used for political ends. Findings attributed to race actually often reflect cultural 
or socioeconomic factors. It is important to remember that even widely used variables and 
response choices should be examined in terms of whether they will provide information in ways 
most appropriate for local purposes. 

Primary language/immigration status. Language difference is another variable that may help 
State planners design programs that are appropriate to the populations they serve. The 
Washington State system includes a question asking whether use of English is functional or 
limited, as well as an item identifying the primary functional language from among a list of 46 
languages, including American Sign Language. Recent immigration status may also be important 
for cultural considerations and special needs, particularly in those States with large numbers of 
immigrants. 


Some persons argue against collecting data on race because results can be misinterpreted or used for 
political ends. Findings attributed to race actually often reflect cultural or socioeconomic factors. 


Education 

The Federal Client Data Set uses the highest school grade completed as a measure of education 
level (a general equivalency diploma is equivalent to 12 years). Another option is a categorical 
variable that includes as response choices the highest degree attained: none, high school diploma, 
vocational technical certificate, bachelor's degree, master's degree, or professional degree (e.g., 
M.D., J.D., or other doctoral degrees). 

Social History 

Marital/relationship status, parenthood, sexual orientation. Marital status is an optional item 
on the Federal Client Data Set. The item as it exists does not provide a category for persons 
living in a committed relationship but not legally married. OMS planners might want to consider 
different response choices for this existing item or perhaps add an additional item. 

Information on parenthood status and number of dependents can also help with program design 
and may be predictive of outcomes. The Washington State forms include questions about the 
number of children living with the patient (their own and others'), as well as children not living 
with the patient. 







Sexual orientation can be a controversial item, and there may be doubt about the truthfulness of 
patients' responses because of their fears of discrimination. Some proponents argue that it is 
important to determine whether gay, lesbian, bisexual, or transgender individuals are more likely 
than others to have AOD problems. It is important to remember, of course, that treatment 
admissions are not an accurate reflection of AOD abuse prevalence. If underreporting were to 
occur, it would distort the findings anyway. As with all other variables under consideration, the 
key question is utility. Is the question asked routinely as part of individual treatment planning? 
Would the system be changed in any way as a result of findings in this area? If the data are not 
going to be put to practical use, it is probably not justifiable to ask patients to reveal such 
sensitive infonnation. 

Living arrangement. One simple breakdown is provided as an optional item in the Federal 
Client Data Set: homeless or transient, dependent living (as with parents or in a supervised 
setting), or independent living. These latter two response choices could be broken down into 
subcategories. For instance, dependent living could include separate response choices for living 
with parents, living with other relatives, board and lodging, halfway house, and so forth. 
Independent living could include separate choices for house, apartment, or mobile home. The 
utility of more detailed information should guide the decision process. 

Veteran status. Veteran status is also an optional item in the Federal Client Data Set. The item 
is easy to collect and noncontroversial but may have limited utility overall. Minnesota modified 
the item so the "Yes" response was changed to "Yes, no combat" and "Yes, served in combat 
zone" to see whether combat experience might be a predictor variable. Iowa has modified this 
question to address military status, with response choices of "none," "veteran," "in reserves," and 
"active duty." 

Socioeconomic status, income, health insurance. Socioeconomic status (SES) is a useful 
concept but not necessarily an easy one to operationalize. OMS planners may want to review 
available SES classifications to detennine whether any would be useful for their purposes. 
Several optional items from the Federal Client Data Set can be used as rough SES indicators, for 
example, primary source of income/support, health insurance, and expected source of payment 
for treatment. Other options include personal or family income and occupational level. The State 
of Washington asks for monthly household income and monthly personal income. Iowa asks for 
taxable individual monthly income as well as occupation level (professional/managerial, 
sales/clerical, crafts/operatives, nonfarm laborers, farm owners and laborers, and 
service/household). 

Health insurance variables can be useful since health insurance plays a part in dictating the kind 
of healthcare to which a person has access, which may have a significant relationship to 
outcomes. While any one variable in this general area has its limitations, a selection of several of 
these variables will probably provide sufficient information for categorizing patients. 

Spirituality/religious affiliation. Another factor that can be considered as part of social history 
is religious affiliation and spirituality. Recovering AOD abusers often emphasize the importance 
of spirituality in their recovery. Involvement in church activities is another predictor of positive 



outcomes. Many AOD treatment programs use the 12-step approach, and issues of religiosity and 
spirituality may be important and can be measured (Tonigan et ah, 1991). 

Family and Other Social Relationships 

The ASI addresses several aspects of family and social relationships. As lifetime measures, it 
determines whether close personal relationships existed with parents, siblings, partner(s), 
children, and friends. It also asks about periods of serious problems with these people as well as 
with other family members, neighbors, and coworkers. (This section, like all others in the ASI, 
also includes many 30-day measures for baseline/outcome comparison.) 

Other variables could certainly be used. The key question, in terms of both treatment planning 
and predictive value, would be the availability and extent of social supports. 

History of physical/sexual abuse. Many adolescents and adults in treatment report histories of 
sexual abuse and family violence. It is not yet known to what extent these factors may be 
predictors of outcome. Documentation of the extent of these problems among AOD patients, 
however, can be used for treatment planning and design purposes. 

Many instruments, including the ASI, require a yes or no response to questions about abuse, 
victimization, and/or perpetration. For example, the Washington questionnaire asks whether the 
patient has ever been a victim of domestic violence and whether the patient is currently a victim 
of domestic violence. While this format allows for quantification of the extent of these problems, 
it may be inadequate as a predictor of treatment outcome. Many factors contribute to the impact 
of physical and sexual abuse on its victims: identity of the perpetrator, age of the victim, 
frequency and severity of abuse, and whether the victim sought or received help at the time or 
since the abuse. Assessment of all these issues, while appropriate for clinical purposes, would be 
beyond the scope of an OMS. 

Abuse victimization and perpetration can also be used as baseline/outcome measures and will be 
discussed in this context later in this chapter. 

Vocational History 

Employment status. A variable that can present problems is employment status. The Federal 
Client Data Set includes a forced-choice response format: employed full time, employed part 
time, unemployed (and looking for work), or not in the labor force. This last category can be 
expanded into another forced-choice format that includes homemaker, student, disabled, retired, 
and correctional inmate. A forced choice results in the loss of information for patients who have 
multiple roles and responsibilities, such as the homemaker who is employed and goes to school, 
or the student who is also employed. Multiple role status in itself may be a predictor of outcomes 
and thus valuable information to retain. Forced choices by their very nature reflect a value as to 
which is more important when multiple responses apply, a value which may not be shared by 
patients or other OMS stakeholders. In the case of the Federal Client Data Set, "employed full 
time" would supersede choices listed below, and other applicable information would be lost. 


Loss of information can be avoided with a "check all that apply" format. For Federal reporting 
purposes, the responses can be recoded into the hierarchy of response choices set forth in the 
Client Data Set guidelines. While this TIP cannot address the nuances of all variable options in 
this detail, the employment status variable, like the race variable, points out the pitfalls of 
assuming that demographics are easy variables to define or agree on. 

Employment history. Current employment status does not provide information about 
occupational stability over time or give any indication of downward mobility that might be 
associated with AOD use. The ASI includes variables on length of longest held full-time job, 
most recent or current occupational status, and usual employment pattern over the previous 3 
years. The Iowa system inquires about the number of months worked during the previous 6 
months. These are examples of variables that inquire beyond employment status at admission. 


Current employment status does not provide information about occupational stability overtime or 
give any indication of downward mobility that might be associated with AOD use. 


AOD Use History 

Information about the history of substance use is important for proper evaluation of treatment 
outcomes because the type of drug and the severity of the AOD problem may dictate the 
intensity and specificity of treatment as well as predict outcome. People with a substance use 
disorder are a heterogeneous population with a variety of different use patterns; however, the 
effects of all these differences on outcomes are unclear. 

Because reduction in substance use and its associated consequences is one of the primary goals 
of treatment, many substance use variables are appropriate as baseline/outcome measures and 
will be discussed later in this chapter. However, some substance use variables can be used as 
predictors as well, and some are solely one-time measures. 

Substances used, primary drugs of abuse. Some indication of the substances used by 
individual patients is essential to an OMS. There are various ways to collect this information, 
and several combine type of drug (or drug class) with use frequency. The Federal Client Data Set 
requires coding the drug (or drugs) identified as primary problem, secondary problem, and 
tertiary problem for each patient. Clinical judgment is used to make the determination. For 
primary and secondary drugs, codes are also required for age of onset of use (first intoxication 
for alcohol), frequency of use, and usual mode of administration. This data set has some 
limitations, which will be discussed below. 

Age of onset. Age of onset of use has been shown to be a predictor of substance use severity in 
both epidemiological and clinical studies (Buydens-Branchey et ah, 1989). In the Federal Client 
Data Set, this variable is required only for primary and secondary drugs of abuse. If the patient 
initiated use with a third drug, the required information does not provide actual age of onset of 







any AOD use. This limitation can be resolved by either asking for this directly or including age 
of onset for all drug or drug categories reported by the patient. 

Frequency of use. In the Federal Client Data Set, this variable is also required only for primary 
and secondary drugs of abuse. If the patient uses a variety of drugs, no information is obtained 
on the remainder. The variable is further limited by focusing only on recent use. A solution to 
both these problems is to add frequency of use as a variable for all drugs reported. It may be 
preferable, however, to exercise this option as a baseline/outcome measure so that frequency of 
use over a specified time interval can be compared before and after treatment. Use frequency will 
be discussed in greater detail in the section on baseline/outcome measures. 

Mode of drug administration. "Usual mode of administration" is included in the Federal Client 
Data Set. Because it is required only for primary and secondary drugs of abuse, it does not serve 
to identify all current (or previous) injection drug users. It only identifies those clients whose 
usual mode of administration is injection. This information may be deemed inadequate, 
considering the risks for HIV infection associated with even infrequent use, or previous use 
dating back many years. Several States, including California, Iowa, Minnesota, and Washington, 
have included a more general question to address any history of injection drug use. This question 
need not be drug specific. 


Age of onset of use has been shown to be a predictor of substance use severity in both 
epidemiological and clinical studies. 


AOD diagnoses. Substance use disorder diagnoses recorded at intake or discharge may be useful 
for purposes of population profiles. The Diagnostic and Statistical Manual of Mental Disorders, 
Third Edition, Revised (DSM-III-R) (American Psychiatric Association, 1987) codes are optional 
variables in the Federal Client Data Set. However, diagnostic categories alone are of limited 
value as predictor variables and should be supplemented by other indicators of problem severity 
such as frequency of use and use consequences. The full series of AOD abuse/dependence 
diagnostic codes should be listed with instructions to "check all that apply" in order to capture 
the full range of applicable diagnoses for poly drug-using patients. 

Treatment History 

Treatment history is an important patient variable to measure. It can provide an efficient 
indicator of the severity of a patient's alcohol or drug problem. The number of lifetime treatment 
admissions and number of lifetime detoxification admissions, for example, can assist with 
treatment placement decisions and may be useful predictors of outcome. The number of prior 
AOD treatment admissions is included in the Federal Client Data Set. Iowa limits the period of 
inquiry to 10 years. 







Medical History 


In many studies, a relationship has been found between AOD use and injuries, physical 
disabilities, and other medical problems (Cherpitel, 1992; Cregler, 1989; Eckardt et ah, 1981; 
Heinemann, 1993; Moessner, 1979). Whether to collect information on medical history and 
current medical conditions and the amount of detail to include will depend on the purposes of the 
OMS and the availability of resources. Background information could include such data as 
number of lifetime hospitalizations (excluding childbirth) or a general rating scale of overall 
physical health. On the other hand, if more detail is sought, questions could be asked about the 
nature and duration of illnesses or symptom severity. 

While it is true that many illnesses are associated with substance abuse, the purpose of collecting 
exhaustive medical data in an OMS must be justified. What might be appropriate for a medical 
workup is not necessarily appropriate for an outcomes monitoring system. This is another area 
that could generate an almost unlimited number of potential variables, and decisions must be 
guided by the utility of the information in analyzing treatment outcomes. If the purpose is to 
determine whether ill health is related to outcomes, general indicators will probably suffice. If 
the purpose is to determine to what extent medical problems remit with treatment, a 
baseline/outcome measure would be needed. 


The purpose of collecting exhaustive medical data in an OMS must be justified. What might be 
appropriate for a medical workup is not necessarily appropriate for an outcomes monitoring system. 
Decisions must be guided by the utility of the information in analyzing treatment outcomes. 


The ASI includes questions about lifetime medical hospitalizations, chronic medical problems, 
and regular prescription drug use. The Washington system includes questions covering numbers 
of emergency room visits, outpatient/clinic visits, inpatient hospitalizations, and days of 
hospitalization in the previous year. Washington also has separate questions that address whether 
or not the patient is currently under medical care for infectious disease, traumatic injury, 
continuing illness, or dental problems. 

Pregnancy status. Pregnancy status is an optional and controversial item in the Federal Client 
Data Set. In some States, legislation has been passed that mandates reporting certain illicit drug 
use by pregnant women to local authorities. This requirement may motivate women to deny 
pregnancy. Depending on whether this item was completed at intake or shortly thereafter, 
pregnancy may also be underreported. Some women may discover they are pregnant at a later 
point during treatment (or not know at all). 

Psychiatric History 

Psychiatric disorders are found in AOD-abusing populations at rates higher than in general 
populations (Helzer and Pryzbeck, 1988; Hovens et ah, 1994; Regier et al., 1990; Rounsaville, 
1990; Stowell and Estroff, 1992). Psychiatric disorders may predate AOD abuse, be a 







consequence of AOD abuse, or coexist with AOD abuse. Psychiatric disorders include 
depression and bipolar disorder (manic-depression), anxiety disorders, schizophrenia and other 
thought disorders, posttraumatic stress disorder, dissociative disorders, and eating disorders and 
other compulsive behavior disorders. They also include personality disorders such as antisocial 
personality disorder. (For more information about dual diagnosis of psychiatric disorders and 
AOD abuse, refer to another TIP in this series, Assessment and Treatment Planning for Patients 
With Coexisting Mental Illness and Alcohol and Other Drug Abuse.) 

As with physical disorders, it is important to be selective about the amount of infonnation 
collected about psychiatric disorders and emotional distress. Collecting and recording data for an 
OMS is not the same as clinical assessment or documentation. Again, the guiding principle in 
OMS planning is to select those variables that may be significantly associated with treatment 
outcome or that serve as baseline/outcome measures. For example, the amount of subjective 
distress experienced by a patient may have a greater relationship to outcome than to the specific 
nature of the psychiatric diagnosis, or a combination of the diagnostic code and clinical severity 
rating may provide the most useful information. Ultimately, for purposes of data analysis, large 
numbers of variables must be distilled into a smaller number of useful ones. A review of existing 
studies and outcome monitoring systems may guide the distillation process. Incorporating a 
variety of psychiatric assessment instruments into the OMS may seem like a safe way to be sure 
all the bases are covered, but it is also likely to pose an insurmountable obstacle in terms of staff 
and patient time and data analysis. 


The amount of subjective distress experienced by a patient may have a greater relationship to 
outcome than to the specific nature of the psychiatric diagnosis, or a combination of the diagnostic 
code and clinical severity rating may provide the most useful information. 


The Federal Client Data Set includes an optional variable that merely records the existence of a 
psychiatric problem. This question is probably too general to be of any predictive value and too 
vague to provide useful interpretation of findings. Washington includes an item to record 
whether or not a psychological evaluation was conducted and a psychiatric diagnosis was made, 
as well as yes/no questions on the current use of mental health services and psychotropic 
medication. 

The ASI includes questions on the number of lifetime psychiatric hospitalizations and courses of 
outpatient treatment. The ASI also asks about lifetime episodes of depression, anxiety, 
hallucinations, memory or concentration difficulties, difficulty controlling violent behavior, 
suicidal thoughts, and suicide attempts, as well as the use of psychotropic medication. The 
Minnesota OMS added a question to the ASI series to address other compulsive behaviors such 
as eating disorders and gambling. Similarly, Iowa asks a general question about addictions other 
than AOD dependence. 







Motivation to Enter Treatment, Treatment Readiness, and Coercive Influences 

Patients enter treatment for many reasons. Factors that influence the decision may be internal or 
external, and for many patients a variety of factors may impinge on the decision. For some, the 
admission is involuntary; they are court ordered or committed to treatment. For others, coercion 
may be more subtle: the threat of a spouse's or partner's leaving or the loss of a job, a 
professional license, or custody of children are often significant factors. Patients may also enter 
treatment because they are sick, broke, or weary of maintaining their addiction. 

An OMS may include a measure of the coercive influences and motivations to enter treatment in 
order to determine their relationship to treatment outcomes. Findings in this area could have 
important policy implications and could be used to improve patient outcomes. A rating of patient 
readiness for treatment could also be included. If patient readiness were found to be a strong 
predict or of positive outcomes, it might indicate a change in patient placement strategies or the 
need to search for ways to improve patient readiness. 

In Minnesota, the OMS includes a question that addresses the primary condition surrounding 
admission to treatment. Response choices include treatment versus incarceration or as a 
condition of probation or parole, to avoid loss of children or to regain their custody, to prevent 
loss of a relationship or living situation, to regain a driver's license, to keep a professional license 
or job or to stay in school, to retain eligibility for government benefits, or to escape financial 
pressures related to continued alcohol and drug abuse. 

Baseline/Outcome Variables 

The key to valid and meaningful baseline/outcome measures is a comparable interval of 
measurement before and after treatment; Chapter 4 addresses this issue in detail. The advantages 
of short windows, such as 30 days, are more accurate recall and finer precision of measurement. 
A drawback is the lack of information for a longer duration. In fact, this lack is one of the 
criticisms of the ASI, which measures AOD use frequency only for the previous 30 days. These 
questions, asked 6 months after discharge from treatment, would not reveal whether the patient 
had used any alcohol or other drugs since treatment. In contrast, a 6-month interval covers a 
longer duration but lacks precision in terms of measurement. The recommended solution is a 
combination of both short-term and long-term intervals. This combination will be explained 
more in the discussion of specific variables that follows. 

Frequency of AOD Use 

The most obvious baseline and outcome measures are those associated with substance use and 
abuse before and after treatment. Measures of frequency of AOD use are almost always included 
in treatment outcome studies. Frequency of use is important to measure because it is associated 
with a range of biopsycho-social problems. Reduction of use frequency is a useful outcome 
measure even in the absence of total abstinence. A number of instruments incorporate effective 
ways to ask about frequency of AOD use. The ASI includes use frequency for a variety of drugs 
and drug categories but limits the inquiry to the previous 30 days. 


Measures of frequency of AOD use are almost always included in treatment outcome studies. 
Frequency of use is important to measure because it is associated with a range of biopsychosocial 
problems. 


Minnesota's OMS design addressed the limitations of both the Federal Client Data Set and the 
ASI in measuring AOD use frequency. For all the Client Data Set drug categories, use frequency 
responses are: 

• No use in past month 

• 1-3 times in past month 

• 1-2 times per week 

• 3-6 times per week 

• Daily. 

The Minnesota OMS replaces the "no use in past month" response choice with three other 
choices: 

• Used in past 2-6 months (but not in past month) 

• Used in lifetime (but not in past 6 months) 

• Never used. 

This design captures the information as required for the CDS but also allows for a record of 6- 
month and lifetime use. (The lifetime response choice is used only at admission, not at 
followup.) 

AOD Use Amounts 

The amount of drug use per typical occasion of use can be an indicator of the severity of the 
AOD problem and the individual's ability to maintain control over use. Amount of use is not 
used as often as frequency of use because it is more difficult to quantify and typically less valid 
and reliable. For example, while quantity of alcohol consumed during a typical drinking occasion 
may be a useful measure for social drinkers, its value diminishes for persons who abuse or are 
dependent on alcohol. Alcoholics may drink from the bottle, so a measure of number of "drinks" 
becomes meaningless. Even when persons drink cans of beer, glasses of wine, or mixed drinks, 
their accuracy in reporting quantity is likely to decline with increased consumption (Webb et ah, 
1991). 

Amount of use of illicit drugs is even more difficult to quantify. There is no standard measure of 
drug use comparable to that for alcohol. Potency is not controlled by governmental regulation. 
Amount of consumption (number of joints, hits, lines, or fixes) may provide some indication of 
the compulsiveness of the drug-taking behavior, but it does not indicate how much of the drug 
was actually ingested. Drug quantity is not recommended as an OMS measure. 








The amount of drug use per typical occasion of use can be an indicator of the severity of the AOD 
problem and the individual's ability to maintain control over use. 


Mode of Drug Administration 

Mode of drug administration is an important variable for a number of reasons. Injection drug use 
is an indication of problem severity and also important in terms of the risk of transmission of 
HIV and other infectious diseases. The Client Data Set includes "usual mode of administration" 
at admission and the same questions could be asked at followup. Another option is to specifically 
ask for the frequency of injection (the mode of administration of greatest concern) for an 
identical time interval before and after treatment. 

HIV Risk Behaviors 

Injection drug use has already been addressed. In Minnesota, another question addresses the 
frequency of using "clean works" among patients who report injection drug use. High-risk sexual 
behaviors have also been associated with AOD use, either because of poor judgment under the 
influence or the use of sex as a commodity to obtain drugs. Questions can address the use of 
condoms during sexual intercourse and the number of sexual partners over a specified interval 
before and after treatment. 

These are obviously highly sensitive topics. In Minnesota, where questions about using clean 
works are included in the OMS, these questions are the ones most likely to draw objections from 
former patients during followup telephone interviews. Nonetheless, they may prove useful 
measures of behavior change. 

AOD Use Dependence Symptoms 

Measures of symptoms of dependence are useful in detennining the severity of a relapse 
following treatment. Dependence symptoms include: 

• Loss of control over use 

• Unsuccessful attempts to control use 

• Tolerance 

• Withdrawal symptoms. 

According to the fourth edition of DSM (American Psychiatric Association, 1994), a remission 
has been achieved when abuse and dependence symptoms have been absent at least 1 month. 
Early remission covers the first 12 months without symptoms, and sustained remission covers the 
period beyond 12 months. Remission can be further categorized as partial or full. In full 
remission, no abuse or dependence symptoms are present; in partial remission, full criteria for 
dependence are not met, but at least one criterion is present either intermittently or continuously. 







Including DSM-IV criteria symptoms in the OMS will allow for classification of patients’ 
posttreatment status in terms of partial or full remission (or partial or full recovery). 

The Federal Client Data Set includes diagnostic codes from the third revised edition of DSM as 
optional items. (In DSM-IV, the latest edition of DSM, the diagnostic codes for abuse of and 
dependence on particular substances are the same as in the third revised edition.) However, 
diagnosis is likely to be established in a clinical setting as a routine part of the assessment 
process. Including all the relevant questions to establish diagnosis at followup may be too time 
consuming, relative to the benefits of this particular element. The Minnesota OMS limits inquiry 
in this area to a question on the number of times the patient experienced withdrawal symptoms in 
a 6-month period before and after treatment. 

Physical Health Measures 

AOD use can have direct and indirect effects on physical health. Direct effects include, for 
example, infections from injection drug use and damage to the liver and other organs from 
alcohol. Indirect effects result from poor nutrition, unsanitary living conditions, lack of sleep, 
and other health risks associated with AOD use and AOD-related lifestyles, such as traumatic 
injury. 

It is sufficient for purposes of an OMS to measure health status before and after treatment, 
without necessarily linking health status to AOD use. For example, the ASI asks, "How many 
days have you experienced medical problems in the past 30 days?" as well as "How troubled or 
bothered have you been by these medical problems in the past 30 days?" and "How important to 
you now is treatment for these medical problems?" Reduction in problem days or subjective 
severity rating can be used to show improved status after treatment. 


It is sufficient for purposes of an OMS to measure health status before and after treatment, without 
necessarily linking health status to AOD use. 


Other options are questions that record the number of days in the hospital or visits for emergency 
care during, for example, a 6-month period before and after treatment. 

General medical care may actually increase for a time during or after treatment as patients attend 
to long-neglected health issues. For this reason, measures of outpatient physician or clinic visits 
do not always show a posttreatment reduction. Inpatient and emergency care, in contrast, 
typically show dramatic decreases (California Department of Alcohol and Drug Programs, 1994; 
Harrison and Hoffmann, 1989). 

Psychological Health Measures 

As with physical health, AOD use can have direct and indirect consequences for psychiatric 
status and emotional well-being. AOD use can exacerbate existing mental health problems or 







bring on depression and difficulties with memory or concentration. Living conditions and 
disrupted interpersonal relationships associated with severe addiction can also cause emotional 
distress. Finally, difficulty controlling violent behavior, suicidal thoughts, and suicide attempts 
can also be associated with AOD use. 

The ASI asks about a variety of psychological symptoms over the previous 30 days, information 
which provides useful baseline/outcome measures. The psychological questions parallel the 
medical questions regarding number of days (in the past 30) on which emotional problems were 
experienced, a subjective rating of problem severity, and a rating of the need for treatment. 

Psychiatric hospitalization over a 6-month period or longer could be used as a baseline outcome 
measure; however, base rates are relatively low, limiting the utility of this item. Outpatient visits 
may actually increase as patients follow through on treatment plans that include attention to 
mental health as well as AOD use. 

Employment Measures 

Current employment status, a Federal Client Data Set item, can be used at admission and at the 
posttreatment contact. Months of employment in the preceding 6 months is another option. 

The ASI uses a 30-day measurement window and several variables: days paid for working, net 
employment income, days experiencing employment problems, subjective rating of employment 
problem severity, and rating of the need for counseling for problems related to employment. 

Financial Stability Measures 

Relying solely on employment variables to assess financial stability is inadequate because a large 
number of patients may not be in the work force (such as students, homemakers, retired, and 
disabled persons). Measures of financial stability can be used to indicate change in status before 
and after treatment. The ASI includes 30-day income measures for unemployment compensation, 
public assistance, and pensions or social security. 

The Minnesota modification of the ASI incorporates questions about financial stability 
comparable to the employment questions: days experiencing financial problems, subjective 
rating of financial problem severity, and rating of the need for assistance with financial 
problems. 

Legal Problem Measures 

Legal problems can be quantified by using measures such as number of arrests over a specified 
interval before and after treatment. Researchers with experience in this area often prefer arrests 
to actual crimes because arrests are a matter of public record, making it more likely that patients 
will give an honest accounting. A recent California study was very successful at quantifying 
actual crimes (California Department of Alcohol and Drug Programs, 1994), but the typical 
OMS is not likely to have comparable resources available. 


Legal problems can be quantified by using measures such as number of arrests over a specified 
interval before and after treatment. 


The ASI uses two 30-day measures: number of days detained or incarcerated and number of days 
engaged in illegal activities for profit. It also includes the patient's subjective rating of severity of 
legal problems and need for counseling or referral for those problems. 

Family/Social Relationships 

Marital or relationship status can be used at baseline and at outcome, but this information can be 
used to indicate whether the status is an improvement. Living arrangement may also be of some 
limited utility. It may be more helpful to measure satisfaction with marital/relationship status and 
living situation. 

The ASI includes 30-day questions about interpersonal conflicts that provide outcome measures, 
but the absence of conflict may mean merely lack of social contact rather than improved 
relationships. The ASI also includes 30-day measures of sexual, physical, and emotional abuse— 
whether or not the patient has been abused or has abused anyone else. 

The Minnesota modification of the ASI adds 30-day measures regarding family stressors other 
than conflict. It also includes a rating of how troubled the patient has been by loneliness or being 
alone, as well as questions about living with people who use alcohol or other drugs or who have 
AOD problems. 

The limitation of these instruments is that neither the ASI nor Minnesota's modification 
addresses changes in positive aspects of family or social relationships, especially the 
establishment of new relationships by patients who had previously been isolated or estranged 
from others. 

Treatment Variables 

Much remains unknown about how outcomes relate to the type and quantity of treatment 
received, and this relationship is an issue of concern to the single State agencies. In outcomes 
research, treatment variables have typically been neglected (Sobell et al., 1987), but they have 
important implications for understanding outcomes. Certainly they have important cost 
ramifications, as in the areas of setting and staffing. If different treatment modalities or programs 
can deliver comparable quality of care as measured by comparable outcomes, then cost alone 
might be the basis for choosing one over another. 

If only baselines and outcomes are considered, then treatment itself is the "black box," with what 
goes on within treatment remaining a mystery to outcomes evaluators and policymakers. For an 
OMS to function effectively, there must be a level of specificity about what goes on between 







patient admission and discharge. In the past, for example, researchers have been preoccupied 
with comparing the results of outpatient versus inpatient programs (Cummings, 1991; Miller and 
Hester, 1986), without looking at the actual content of these programs. Issues that must be 
considered include differences in forms of treatment, such as individual, group, and family 
therapy; differences in settings and populations; differences in lengths of stay and frequency and 
intensity of treatment sessions; and differences in the actual components of treatment. 


If only baselines and outcomes are considered, then treatment itself is the "black box," with what 
goes on within treatment remaining a mystery to outcomes evaluators and policymakers. 


Administrative Treatment Variables 

Program Identifier 

An identifier is necessary to distinguish one treatment provider from another. This is a 
noncontroversial item. 

Admission Date, Discharge Date, and Length Of Stay 

For inpatient and residential treatment programs, length of stay can be computed if the admission 
date is recorded on an intake form and the discharge date on a discharge form. For outpatient 
programs, however, this information only indicates length of time from start to finish of 
treatment and gives no indication of number of contacts during that interval. Hours of 
involvement or contact days could be recorded on a discharge form. Service amounts will be 
discussed in more detail later. 

Level of Service 

Level of service is a required item in the Federal Client Data Set. This breakdown or an 
expansion of it should be included in an OMS. The CDS also requires an indication of whether 
an admission is the initial admission in a treatment episode or a continuation of treatment 
resulting from a transfer (such as inpatient to outpatient care). Level of care can also be 
considered a predictor variable. 

Treatment Charges 

Information on costs associated with treatment is essential to an OMS focused on accountability 
for taxpayer dollars. Information could be collected as a package cost for the whole program or 
as itemized costs for specific services. Costs are not readily available to the program staff who 
complete patient forms, however. As some States move to managed care based on capitated 
rates, costs for individual treatment admissions may be difficult or impossible to compute. 







As some States move to managed care based on capitated rates, costs for individual treatment 
admissions may be difficult or impossible to compute. 


Treatment Predictor Variables 

Therapeutic Modality and Social Climate 

Little work has been done in determining the effect of therapeutic focus on patient outcomes for 
different kinds of patients. For instance, while therapeutic communities differ from 12-step 
programs, there is still not enough in formation available to predict which patients would do best 
in which modality. 

Social climate may also affect patient outcomes. For example, a well- kn own program may be 
extremely similar in structure and focus to many other programs, but its high profile may lead 
patients to have greater expectations of success. 

Treatment Components 

While a measure of therapeutic focus might be important, it might be more revealing to include a 
variety of measures of the specific services that comprise the treatment regimen. Most programs 
actually consist of a composite of some of the following components: 

• Individual, group, or family counseling sessions 

• Medical and psychiatric/psychological evaluation and treatment 

• Educational lectures, films, or reading material 

• Dietary and nutritional guidance 

• Education, referral, counseling, testing, and treatment for HIV/AIDS, tuberculosis, and other 
infectious diseases 

• Educational and vocational services 

• Social services including housing, childcare, and transportation 

• Legal assistance 

• Recreational and occupational therapy 

• Independent skills training 

• Biofeedback 

• Acupuncture 

• Psychotropic medication 

• Opioid substitution therapy 

• Exercise 

• Spiritual counseling 

• Others. 

Some treatment components may be specific to cultural heritage and traditional healing 
practices. 








Decisions must be made not only about which treatment components should be measured, but 
also how to measure them. While instruments that attempt to define and quantify treatment 
components are relative newcomers in the field compared with those that assess patient 
characteristics, they should be examined for their usefulness in this area. For instance, a simple 
checklist could be used to indicate whether or not a patient had received a service, while a more 
sophisticated measure could be developed to describe the frequency, intensity, and duration of 
the service. It would be important to know, for example, whether or not a certain minimum 
amount of a service were necessary to achieve any beneficial effect, or conversely, whether or 
not a threshold were reached beyond which there were little or no measurable benefit. While no 
OMS could measure all the potential treatment variables that could be generated, attention to 
selected key variables may produce useful information for allocating limited resources, placing 
patients in the most appropriate settings, and giving them the services most likely to produce 
favorable outcomes. 


It would be important to know whether a certain minimum amount of a treatment service were 
necessary to achieve any beneficial effect, or conversely, whether a threshold were reached beyond 
which there were little or no measurable benefit to the patient. 


The Treatment Services Review (McLellan et al., 1992a) provides a standard measure of the 
nature and number of services provided during the treatment process. Services are categorized 
along the same dimensions covered by the ASI and are recorded weekly. For the Minnesota 
OMS, the Treatment Services Record—a similar instrument—was developed (Flarrison, in 
press). This instrument also uses the ASI dimensions to categorize services and quantifies hours 
in treatment and in specific components of treatment (educational sessions; individual, group, 
and family counseling sessions; support group meetings; and informal contact with other 
patients). Both the Treatment Services Review and the Treatment Services Record measure 
contacts that are part of formal treatment as well as ancillary services, since both may affect 
outcomes. The Iowa OMS also quantifies time involved in specific services, recorded on a 
monthly basis. 

Staffing Patterns 

Staffing patterns can be described by considering several objective and subjective qualities of 
staff members. These include the following: 

• Professional training and availability of staff with specialized training to do specialty work 

• Recovery status and prior AOD involvement of counseling staff 

• Therapist characteristics such as empathetic versus confrontational, directive versus facilitative 

• Management style 

• Staff morale 

• Amount of staff contact with patient. This contact may vary considerably in both duration and 
intensity. 








Few data exist on the relation of staffing patterns to patient outcomes. Selectivity in measuring 
the staff qualities listed above would be imperative if staff qualities were to be included as 
measures in an OMS. Most patients are likely to have contact with a variety of staff, so they may 
be exposed to a variety of levels of expertise, AOD use histories, and therapist characteristics. In 
addition, staff characteristics may not be static and may interact in different ways with individual 
patient characteristics. Although staffing characteristics and staffing patterns may be related to 
treatment outcomes, a decision might be made not to include these measures in an OMS because 
of lack of data about the benefits of collecting this infonnation in relation to the time and costs 
involved. 

Context/Special Populations Served 

Programs vary tremendously in terms of the populations they serve. Special populations can be 
defined by personal characteristics such as gender, age, race/ethnicity, cultural background, or 
sexual orientation. Because of Federal block grant mandates, many States are currently designing 
more programs for pregnant women and women with children. Special populations can also be 
defined by drug use patterns; some programs are designed specifically for people addicted to 
heroin or cocaine. Programs can also be designed for individuals with coexisting disorders such 
as mental illness, eating disorders, or compulsive gambling. If an OMS were to find that certain 
patients did better in programs designed to serve their special needs, compared with other 
settings, these results could be used to justify increased funding for such targeted services, 
despite the increased costs these might entail. 

Discharge Status 

The patient's treatment completion status is essential information to collect in an OMS. 
Successful treatment completion can be used as an intennediary outcome measure (Wickizer et 
ah, 1994), whereas leaving against staff advice may predict a poorer posttreatment outcome. 
Other reasons for failure to complete treatment may provide useful information to improve 
treatment retention. 


The patient's treatment completion status is essential information to collect in an OMS. Successful 
treatment completion can be used as an intermediary outcome measure, whereas leaving against 
staff advice may predict a poorer posttreatment outcome. 


Patient Satisfaction 

Patient satisfaction with treatment as a whole or with specific services received can be measured 
during and after treatment. Satisfaction ratings by service type are included in the Minnesota 
weekly Treatment Services Record as well as in the followup interview. The Iowa system also 
includes patient ratings of the benefits of certain treatment components collected at followup. 
While patient ratings of treatment may not be related to other treatment outcomes, they may be 
important to consider in an era that is becoming more attuned to customer satisfaction. 







Satisfaction may also be related to treatment retention, so a correlation between patient 
dissatisfaction and early dropout from treatment may provide useful information about program 
changes necessary to keep patients in treatment. 

Posttreatment Services 

The use of community resources following discharge from treatment may also have an impact on 
patient outcomes. Patients are typically referred to peer support groups and other available 
resources to assist with their recovery following discharge from formal treatment. Learning more 
about the relation of these services and successful treatment outcomes can assist with discharge 
planning and resource allocation. 


Existing State Systems 

The content included in existing State outcomes monitoring systems provides a basis for review 
by States planning a new system or revising their current systems. Several State systems, in 
varying stages of development, are described in Appendix B. A contact resource is listed for 
each, to aid readers in obtaining a set of current instruments and other relevant information. 
Because of their length and because revisions occur periodically, individual forms are not 
reproduced in this TIP. 

All the discussion of OMS content in this chapter is based on experience to date. In this rapidly 
evolving field, recommendations will change as new information emerges from existing systems 
and ongoing studies. Some of the data elements currently in widespread use may be found to be 
of limited utility in evaluating outcomes. Others, in trial stages or not yet developed, may prove 
to offer more benefits. While this TIP can provide a useful overview of outcomes monitoring 
systems and general guidelines for content, decisions for an individual State will depend on local 
needs and an up-to-date review of continuing improvements in this field. 


Exhibit 6-1 

Consent to Participate in Alcohol or Other Drug Abuse 
Treatment Research, Evaluation, and Followup 
Consent for the Release of Confidential Information 

I,_[name of patient], consent to 

participate in alcohol and other drug abuse treatment research, evaluation, and followup. I 
understand that I will be interviewed by 

_, a [clinician/ 

independent researcher/State employee], during my time in treatment about my problems 
and the services I receive. I understand that I will also be contacted and interviewed after 

I finish treatment by_, 

a [clinician/independent researcher/State employee] and asked about my progress since 
treatment. 

For purposes of participating in this research, I authorize 


[name of treatment program] to disclose my name to 

_[name of agency 

conducting the outcomes monitoring systems study] so that it can have someone 
interview me. 

I am providing an address and telephone number where I believe I can be located in the 
future, and the names, addresses, and phone numbers of others who may be of help in 
locating me. I understand that these persons will be contacted only concerning my 
whereabouts and that nothing about my treatment or my condition or the fact that I was in 
treatment will be disclosed to them or anyone else. 

I understand that my records and the information I provide are protected under the 
Federal regulations governing Confidentiality of Alcohol and Drug Abuse Patient 
Records, 42 CFR Part 2, and cannot be disclosed without my written consent unless 
otherwise provided for in the regulations. I also understand that I may revoke this consent 
at any time except to the extent that action has been taken in reliance on it, and that in any 
event this consent expires as follows: 


Specification of the date, event, or condition upon which this consent expires 
Dated: 


Signature of patient 


Signature of parent, guardian, or 

















authorized 

representative when required 


Followup Research 

Research that follows patients for any period after they leave treatment presents special 
challenges. First, under the Federal regulations, no information the researcher or evaluator 
gained from the AOD program with the patient's consent or through the research, audit, and 
evaluation exceptions may be disclosed to anyone else. Yet, the researcher must locate the 
patient in order to collect followup data. Second, any research that can be perceived as imposing 
a burden on patients (for example, the burden of submitting to interviews) or that may require the 
researcher to contact either collateral sources or persons who can help locate patients in the 
future will require patients' informed consent to participate. This question is discussed first. 

Obtaining Informed Consent 

The best practice is to obtain patients' informed consent to participate in research that will 
impose a burden on them or that may require the researcher to contact others to inquire about the 
patient. This kind of consent is different from the consent discussed above, which authorizes an 
AOD program or researcher to disclose information to a third party. 

Obtaining informed consents from patients ensures that they agree to participate in the research 
project; that they understand that, as part of the project, efforts will be made to locate them 
through the persons whose names they submit (or that they or collateral sources will be 
interviewed); and that they appreciate the risks of participation (see Exhibit 6-2). In research that 
involves followup, the risk to the patient is most likely limited to a disclosure that he or she was 
in AOD treatment. Obtaining informed consent does not authorize a researcher to make a 
disclosure ofpatient- identifying information to third parties. As will be discussed below, if 
followup research is designed and conducted with care, it need not involve disclosures to third 
parties. If disclosure will be made to third parties, the researcher must obtain patients' consent in 
accordance with §§2.31. 

Conducting Followup Research 

Locating and interviewing former patients without making any prohibited disclosures. To 

ensure that patients can be located months or years after they leave treatment, researchers 
sometimes ask for the names of persons with whom the patients are likely to have continued 
contact. Making inquiries in order to locate a former patient might seem at first glance to pose no 
risk to a patient's right to confidentiality; nevertheless, it does. For example, if a research entity 
were to tell a landlord, either directly or by implication, that was looking for Marvin Moe in 
order to see how well he was doing after treatment, it would be letting the landlord know that 
Marvin Moe had been in treatment. The Federal regulations clearly prohibit this kind of 
disclosure, unless the patient consents. Thus, when researchers and evaluators are trying to locate 







a patient, they must do so without disclosing to others any information about the patient's 
connection to AOD abuse treatment. 

If followup contact is to be attempted over the telephone, the caller has to be sure he or she is 
talking to the patient before identifying himself or herself or mentioning a connection to the 
AOD program. For example, asking for Sally Soe when her husband or child answers the phone 
and announcing that the caller is from the Capitol City AOD Program (or the Drug Research 
Corporation) violates the regulations. The program (or research agency) may form another entity, 
without a hint of drugs or alcohol in its name (for example, Health Research, Inc.), that can 
contact former patients without worrying about disclosing information simply by giving its 
name. 

When a representative of such an entity calls former patients, however, he or she still has to be 
careful that the patients are personally on the line before revealing any connection to the AOD 
program. It is a good idea for the research entity to have a set of scripted answers the caller can 
give when questioned by others about the purpose of the inquiry about the former patient. 

If followup is to be done by mail, the return address should not disclose any information that 
could lead someone seeing the envelope to conclude that the former patient was in treatment. 

Followup with collateral sources. Outcomes monitoring that collects data about patients from 
collateral sources raises a similar issue to that raised by followup with patients themselves: How 
can an inquiry be made of relatives (including parents), employers, schools, or social welfare 
agencies without violating the Federal regulations? 

There are two ways to approach this problem. First, the outcomes monitor can structure the data 
gathering to avoid revealing that the patient was in AOD treatment. To accomplish this, the name 
of the entity that conducts the monitoring must be neutral, revealing nothing about an AOD 
connection. The questions asked of the collateral sources must also be drafted so that they offer 
those sources no information that would directly or implicitly link the patient with AOD abuse or 
treatment. 


Exhibit 6-2 

Consent to Participate in Alcohol and Other Drug Abuse 
Treatment Research, Evaluation, and Followup 

You are invited to participate in a study of alcohol and other drug abuse treatment effectiveness. 
The [name of agency conducting study] and treatment programs throughout the State want to 
learn more about what kinds of programs and services help different kinds of clients with 
different kinds of problems. 

What you are being asked to do . 

Each treatment program is asking at least [number ol] clients to participate in this research. If 
you agree to participate, you will be interviewed by a [clinician/independent researcher] about 
the problems you have had and the kind of help you need. The first interview will take 30 to 45 








minutes. Then, each week you are in treatment, you will be interviewed for 5 to 10 minutes to 
record the services you are getting and your opinion of them. 

The final interview will take place 6 months after you leave treatment. You will be 
[telephoned/interviewed in person] by a [clinician/independent researcher/State employee] and 
asked about your progress since treatment. This followup interview will take 10 to 20 minutes. 

During the [first] interview, you will be asked some sensitive questions that might make you 
uncomfortable. You will also be asked about some acts that are illegal, such as drug use. You do 
not have to answer any questions you do not want to answer. However, your answers will be 
kept confidential. 

Other than the discomfort you may feel about some questions, there are no risks associated with 
your participation in this research. There are also no specific benefits for you if you participate, 
but your participation may help treatment programs improve their services. 

All information you provide is confidential . 

All information you provide for this research is confidential and is protected by Federal 
laws and regulations as well as by [name of State statute]. However, the researchers must 
report any abuse or neglect of children. [This requirement varies from State to State.] None of 
the interview forms will have your name on them. Only the consent form attached to this sheet 
will have your name on it, and this will be used only to contact you after treatment for the final 
interview. 

All other forms will use a confidential code number. All the information you provide will be 
stored in a locked [area/cabinet] at [location]. Then, it will be sent to [name of central data 
collection facility] and be entered in a computer. Your name will not be entered in the 
computer. Reports about this research will not identify any individuals in any way. The 
information will be used only to evaluate different kinds of treatment programs and different 
services clients receive. 

Participation is voluntary . 

Participation in this research is voluntary. If you decide to participate, you can change your mind 
at any time. If you decide not to participate, or if you start but change your mind, your decision 
will not affect your treatment in any way. 

If you have any questions about this research now or later, you may call: 

You will be given a copy of this form to keep. 


The second way an entity conducting outcomes monitoring can gather information from 
collateral sources is to obtain the patients' consent to disclose to the collateral source the fact that 
the patient was in AOD treatment. The special consent form required by §§2.31 of the 
regulations must be used to obtain the patient's consent. As outlined above, this fonn requires the 









research entity to state on the consent form the purpose of the disclosure —in this instance, 
outcomes monitoring research—and how much and what kind of information will be disclosed, in 
this instance, the fact that the patient was in AOD treatment. The form must also have an 
expiration date and a statement that consent can be revoked at any time. 

Using a consent form to gather information from collateral sources may require more work 
initially—to obtain consent forms from all patients—but it provides more freedom to the 
researcher. With consent forms from patients, the researcher may ask questions about AOD use. 
However, care must still be taken to reveal only the limited information specified on the consent 
form. The researcher should have a system to keep track of the expiration dates of the consent 
forms being used. 

Tracking patients in social welfare and criminal justice systems. In some cases, outcomes 
monitors will want to detennine, by tracking patients, whether patients reenter the treatment 
system or receive other medical or social welfare benefits or have encounters with the criminal 
justice system. Because of the Federal confidentiality laws and regulations, special care must be 
taken in the design of any research in which an attempt is made to compare the names of patients 
who are receiving or have received AOD treatment with 1) the names of patients who previously 
received AOD treatment or who subsequently enter the treatment system, 2) the names of 
persons receiving other medical or social benefits, or 3) the names of persons arrested for 
criminal activity. 

Clearly, the entity conducting outcomes monitoring cannot simply turn over the names of the 
patients of the programs it is evaluating to other AOD programs, medical or welfare authorities, 
or officials in the criminal justice system with a request that a search be made for the names. 
How, then, can these tasks be accomplished? 

Tracking patients in the AOD system. Tracking whether or not patients reenter the treatment 
system generally requires the creation and continual updating of a database of patients' names. 
Over the course of the study, researchers compare the names submitted by treatment programs to 
the names already in the database. This kind of study can be accomplished in any one of three 
ways: 


• Using patients' consents. All patients can sign consent forms that comply with §§2.31, 
permitting programs to disclose patient identifying information to researchers. The consent 
forms must specify the reason for the disclosure, what information will be disclosed to 
researchers, and an expiration date. (Note that this simple consent form does not permit the 
entity operating the database to disclose information to a treatment program or to anyone 
else.) 

• Using the research, audit, and evaluation exception. If the entity conducting the outcome 
monitoring satisfies the requirements of §§2.52 or 2.53, programs can disclose patient- 
identifying information for entry into and comparison with a database of patients' names. Again, 
no information from one treatment program can be disclosed to another. 

• Using the gualified service organization exception. The Federal confidentiality regulations permit 
AOD programs to enter into an agreement, called a qualified service organization agreement 
(QSOA), when they routinely need to share certain information with an outside agency that 
provides services to them. A QSOA is a written agreement between a program and a person or 



entity providing services to the program, in which that person or entity does the following 
(§§2.11, 2.12(c)(4)): 

1) Acknowledges that in receiving, storing, processing, or otherwise dealing with any 
patient records from a program, it is fully bound by [the Federal confidentiality] 
regulations 

2) Promises that, if necessary, it will resist injudicial proceedings any efforts to obtain 
access to patient records except as permitted by these regulations. 

An entity performing outcomes monitoring is providing a service to the program by studying 
how effective treatment is and how it might be improved. Disclosures made to an entity 
conducting outcomes monitoring with which programs have signed QSOAs must be limited to 
information that is needed to conduct the research. (See sample QSOA form in Exhibit 6-3.) 

• Using the internal communications exception. The Federal regulations permit programs to 
disclose information to an entity that has direct administrative control over them if that entity 
needs "the information in connection with [its] duties that arise out of the provision of 
diagnosis, treatment or referral for treatment of alcohol or drug abuse" (§§2.12(c)(3).). If the 
entity conducting the outcomes monitoring is the single State agency (SSA), the programs it runs 
could disclose the information it needs in order to conduct that monitoring under this 
exception, without patients' consent. However, the SSA could not redisclose patient-identifying 
information to anyone else. 


Exhibit 6-3 

Qualified Service Organization Agreement 

XYZ Service Center ("the Center") and 

the_ 

(name of the program) 

("the Program") hereby enter into a qualified service organization agreement, whereby 
the Center agrees to provide the following services: 

(nature of services to be provided) 


Furthermore, the Center: 












1. Acknowledges that in receiving, storing, processing, or otherwise dealing with any 
information from the Program about the patients in the Program, the Center is fully 
bound by the provisions of the Federal regulations governing Confidentiality of Alcohol 
and Drug Abuse Patient Records, 42 CFR Part 2: and 

2. Undertakes to resist injudicial proceedings any effort to obtain access to information 
pertaining to patients otherwise than as expressly provided for in the Federal 
confidentiality regulations, 42 CFR Part 2. 

Executed this_day of_, 199_. 


President Program Director 

XYZ Service Center (name of program) 

(address) (address) 


Tracking patients in health, social welfare, and criminal justice systems. The Federal 
confidentiality regulations' restrictions on the disclosure and redisclosure of patient-identifying 
in formation generally make it difficult to track whether or not patients seek treatment from other 
medical or mental healthcare providers, apply for welfare benefits, or are arrested by law 
enforcement authorities. As noted above, the entity conducting outcome monitoring cannot 
simply turn over the names in its database to medical or welfare authorities or to officials of the 
criminal justice system, since the Federal confidentiality regulations prohibit this kind of 
disclosure. How can an entity conducting outcomes monitoring compare a database of patients' 
names with a similar database created by the healthcare, welfare, or criminal justice systems? 

A comparison of the names of patients in AOD programs with the names of patients receiving 
other medical or mental healthcare or welfare benefits can be accomplished in one of two ways: 

• If the entity conducting the outcomes monitoring is an SSA that also administers programs and 
has jurisdiction over mental and mental healthcare and/or welfare. 

As noted above, the Federal regulations permit programs to disclose information to an entity that 
has direct administrative control over them if that entity needs "the information in connection 
with [its] duties that arise out of the provision of diagnosis, treatment, or referral for treatment of 
alcohol or drug abuse" (§ §2.12(c)(3)). If the entity conducting the outcomes monitoring is the 
SSA that runs AOD programs and also has jurisdiction over medical and mental healthcare, the 
AOD, medical, and mental healthcare providers under its jurisdiction could disclose the 
information the SSA needs in order to conduct outcomes monitoring under this exception. 
However, the SSA must resist the temptation to use this data for another purpose. For example, it 
could not redisclose the infonnation (names) it receives from AOD treatment providers to other 
AOD treatment providers or to mental healthcare providers or welfare authorities. 

• If the entity conducting the outcome monitoring is an SSA with jurisdiction solely over AOD 
treatment or is any other organization. 












Using patients' consents. All AOD patients can sign consent forms that comply with §§2.31, 
permitting programs to disclose patient-identifying information to the entity conducting the 
research. The consent forms must specify the reason for the disclosure, what information will be 
disclosed to researchers, and an expiration date. Patients receiving care from other medical or 
mental healthcare providers can sign appropriate consent fonns permitting those providers to 
disclose their names to the entity conducting the outcomes monitoring. No patient-identifying 
information that entity receives from AOD treatment programs can be redisclosed. 

Using the research, audit, and evaluation exception for AOD information and patients' 
consents for other information. If the entity conducting the outcomes monitoring satisfies the 
requirements of §§2.52 or 2.53, programs can disclose patient-identifying information for entry 
into and comparison with a database of patients’ names submitted by other medical and mental 
healthcare providers with patient consent. 

Using the qualified service organization exception for AOD information and patients' 
consents for other information. The entity conducting outcomes monitoring could collect 
information from AOD programs by using a QSOA, as outlined above, and from medical care, 
mental health, and welfare authorities by using appropriate patient consent. 

Tracking patients through the criminal justice system is particularly difficult. While an entity 
conducting outcomes monitoring can use any one of the mechanisms listed here to obtain the 
names of AOD patients, it may have difficulty obtaining information from the criminal justice 
system because of State laws protecting the confidentiality of that information. 


Other Issues 

Research and Mandatory Reporting Laws 

If a researcher uncovers child abuse or neglect while reviewing a patient's chart or interviewing 
the patient, must he or she make a report to State authorities? How can a report be made under 
the Federal confidentiality laws and regulations? 

Is the researcher required to report? All 50 States have statutes requiring reporting when there 
is reasonable cause to believe or suspect child abuse or neglect. While many State statutes are 
similar, each has different rules about what kinds of conditions must be reported, 9 who must 
report, and when and how reports must be made. Most States now require not only physicians 
but also educators and social service workers to report child abuse. What is unclear, however, is 
whether or not persons conducting research are mandated reporters. 10 This question is important 
because, as noted below, only those who are mandated by State law to report child abuse and 
neglect may do so under the Federal laws and regulations. 

Because of the variation in State laws, researchers who are concerned about this issue should 
consult an attorney familiar with State law to determine whether they are mandated to report 
child abuse. 


How can a researcher make a report? In 1986, the Federal confidentiality regulations were 
amended to permit AOD programs to comply with mandatory child abuse reporting laws. The 
Federal laws and regulations now permit compliance with State laws that require the reporting of 
child abuse and neglect. However, this exception to the general rule prohibiting disclosure of any 
in formation about a patient applies only to initial reports of child abuse or neglect. Programs and 
others cannot respond to followup requests for information or even subpoenas for additional 
information, even if the records are sought for use in civil or criminal proceedings resulting from 
the researcher's initial report, unless the patient consents or the appropriate court issues a special 
court order (see §§2.64 and 2.65). 

Any researcher contemplating making a child abuse report would be wise to check with the 
program to see if a report has already been made. 

Coding Patients' Identities 

If a researcher codes patients' names to protect their identities, can the intricate rules of the 
Federal confidentiality laws and regulations be disregarded? It depends. As noted above, the 
Federal regulations protect "patient-identifying information." Section 2.11 of the regulations 
defines this to mean the name, address, social security number, fingerprints, photograph, or 
similar information by which the identity of a patient can be determined with reasonable 
accuracy and speed either directly or by reference to other publicly available information. The 
term does not include a number assigned to a patient by a program, if that number does not 
consist of or contain numbers (such as a social security or a driver's license number) that could 
be used to identify a patient with reasonable accuracy and speed from sources external to the 
program. 

Thus, if a research entity can truly code a patient's name so that the number created for each 
patient cannot be "used to identify a patient with reasonable accuracy and speed from sources 
external to the program," it need no longer be concerned with safeguarding information about the 
patient. 


Conclusion 

Outcomes monitoring systems must be designed with patients' interests in mind. Since both 
informed consent and issues of confidentiality can be central to research design, researchers 
should seek the advice of an attorney familiar with these issues to ensure that patients' rights are 
protected. 


Endnotes 

1. This chapter was written for the consensus panel by Margaret K. Brooks, Esq. The Center for 
Substance Abuse Treatment has published another document Confidentiality of Patient Records 


for Alcohol and Other Drug Treatment (Technical Assistance Publication Series 13) that 
addresses these issues. 


2. Hereinafter, citations in this section in the form "§§2..." refer to specific sections of 42 C.F.R., 
Part 2, implementing the Comprehensive Alcohol Abuse and Alcoholism Prevention, Treatment 
and Rehabilitation Act of 1970 (42 U.S.C. §§290dd-3) and the Drug Abuse Prevention, 

Treatment and Rehabilitation Act (42 U.S.C. §§290ee-3). 

3. Violating the regulations by disclosing information is punishable by a fine of up to $500 for a 
first offense or up to $5,000 for each subsequent offense (§§2.4). 

4. Two additional Federal laws permit the U.S. Attorney General and the Secretary of the U.S. 
Department of Health and Human Services (DHHS) to authorize researchers to withhold the 
names and identities of research subjects. The statute authorizing the Secretary of HHS to issue 
confidentiality certificates specifies that it applies to "persons engaged in biomedical, behavioral, 
clinical, or other research (including research on mental health, including research on the use and 
effect of alcohol and other psychoactive drugs)...." Once such authorization is issued, the 
researcher "may not be compelled in any Federal, State or local civil, criminal, administrative, 
legislative, or other proceeding to identify the subjects of research for which such authorization 
was obtained," (42 U.S.C. §§241 (d) permits the Secretary of HHS to issue confidentiality 
certificates; 21 U.S.C. §§872(c) permits the U.S. Attorney General to issue confidentiality 
certificates). 

These statutes may not be particularly useful for those conducting outcomes monitoring because 
most of the infonnation they will gather will already be protected by the Federal confidentiality 
regulations. However, depending upon who gathers followup information from patients, it is 
possible that this information will not be protected by 42 C.F.R., Part 2, as it could be argued that 
the former patients are making voluntary disclosures of information about themselves directly to 
data gatherers. As with many questions about the design of outcomes monitoring, it is best to 
check with an attorney familiar with Federal and State laws and regulations for guidance. 

5. "Patient-identifying information means the name, address, social security number, 
fingerprints, photograph, or similar information by which the identity of a patient can be 
determined with reasonable accuracy and speed either directly or by reference to other publicly 
available information. The term does not include a number assigned to a patient by a program, if 
that number does not consist of or contain numbers (such as social security, or driver's license 
number) which could be used to identify a patient with reasonable accuracy and speed from 
sources external to the program" (§§2.11). 

6. Section 2.16 requires programs to keep written records in a secure room, a locked file cabinet, 
a safe, or other similar container. The program should have written procedures that regulate 
access to and use of patients' records. Either the program director or a single staff person should 
be designated to process inquiries and requests for infonnation. 

7. These particular entities may also copy or remove records, but only if they agree in writing to 
maintain patient-identifying information in accordance with the regulations' security 



requirements (see §§2.16), to destroy all patient-identifying information when the audit or 
evaluation is completed, and to redisclose patient information only 1) back to the program, 2) in 
accordance with a court order to investigate or prosecute the program (§§2.66), or 3) to a 
government agency overseeing a Medicare or Medicaid audit or evaluation (§§2.53(b). 

8. For a discussion of this issue in the context of a consent form permitting interviews with 
collateral sources, see the discussion on "Followup with collateral sources." 

9. For instance, in some States, certain professionals are required to report pregnant women who 
abuse drugs to State authorities. If such a requirement is part of the State's mandatory child abuse 
reporting law, the Federal regulations permit the report. If a separate statute mandates the report, 
the Federal regulations probably do not permit the report. 

10. Some State statutes are worded to require reports only from those persons who have face-to- 
face interaction with a child or adult who discloses abuse. 



Exhibit 6-2 

Consent to Participate in Alcohol and Other Drug Abuse 
Treatment Research, Evaluation, and Followup 

You are invited to participate in a study of alcohol and other drug abuse treatment effectiveness. 
The [name of agency conducting study] and treatment programs throughout the State want to 
learn more about what kinds of programs and services help different kinds of clients with 
different kinds of problems. 

What you are being asked to do . 

Each treatment program is asking at least [number of] clients to participate in this research. If 
you agree to participate, you will be interviewed by a [clinician/independent researcher] about 
the problems you have had and the kind of help you need. The first interview will take 30 to 45 
minutes. Then, each week you are in treatment, you will be interviewed for 5 to 10 minutes to 
record the services you are getting and your opinion of them. 

The final interview will take place 6 months after you leave treatment. You will be 
[telephoned/interviewed in person] by a [clinician/independent researcher/State employee] and 
asked about your progress since treatment. This followup interview will take 10 to 20 minutes. 

During the [first] interview, you will be asked some sensitive questions that might make you 
uncomfortable. You will also be asked about some acts that are illegal, such as drug use. You do 
not have to answer any questions you do not want to answer. However, your answers will be 
kept confidential. 

Other than the discomfort you may feel about some questions, there are no risks associated with 
your participation in this research. There are also no specific benefits for you if you participate, 
but your participation may help treatment programs improve their services. 

All information you provide is confidential . 

All information you provide for this research is confidential and is protected by Federal 
laws and regulations as well as by [name of State statute]. However, the researchers must 
report any abuse or neglect of children. [This requirement varies from State to State.] None of 
the interview forms will have your name on them. Only the consent form attached to this sheet 
will have your name on it, and this will be used only to contact you after treatment for the final 
interview. 

All other forms will use a confidential code number. All the information you provide will be 
stored in a locked [area/cabinet] at [location]. Then, it will be sent to [name of central data 
collection facility] and be entered in a computer. Your name will not be entered in the 
computer. Reports about this research will not identify any individuals in any way. The 
information will be used only to evaluate different kinds of treatment programs and different 
services clients receive. 









Participation is voluntary . 

Participation in this research is voluntary. If you decide to participate, you can change your mind 
at any time. If you decide not to participate, or if you start but change your mind, your decision 
will not affect your treatment in any way. 

If you have any questions about this research now or later, you may call: 

You will be given a copy of this form to keep. 


The second way an entity conducting outcomes monitoring can gather information from 
collateral sources is to obtain the patients' consent to disclose to the collateral source the fact that 
the patient was in AOD treatment. The special consent form required by §§2.31 of the 
regulations must be used to obtain the patient's consent. As outlined above, this form requires the 
research entity to state on the consent form the purpose of the disclosure —in this instance, 
outcomes monitoring research—and how much and what kind of information will be disclosed, in 
this instance, the fact that the patient was in AOD treatment. The form must also have an 
expiration date and a statement that consent can be revoked at any time. 

Using a consent form to gather information from collateral sources may require more work 
initially—to obtain consent forms from all patients—but it provides more freedom to the 
researcher. With consent forms from patients, the researcher may ask questions about AOD use. 
However, care must still be taken to reveal only the limited information specified on the consent 
form. The researcher should have a system to keep track of the expiration dates of the consent 
forms being used. 

Tracking patients in social welfare and criminal justice systems. In some cases, outcomes 
monitors will want to determine, by tracking patients, whether patients reenter the treatment 
system or receive other medical or social welfare benefits or have encounters with the criminal 
justice system. Because of the Federal confidentiality laws and regulations, special care must be 
taken in the design of any research in which an attempt is made to compare the names of patients 
who are receiving or have received AOD treatment with 1) the names of patients who previously 
received AOD treatment or who subsequently enter the treatment system, 2) the names of 
persons receiving other medical or social benefits, or 3) the names of persons arrested for 
criminal activity. 

Clearly, the entity conducting outcomes monitoring cannot simply turn over the names of the 
patients of the programs it is evaluating to other AOD programs, medical or welfare authorities, 
or officials in the criminal justice system with a request that a search be made for the names. 
How, then, can these tasks be accomplished? 

Tracking patients in the AOD system. Tracking whether or not patients reenter the treatment 
system generally requires the creation and continual updating of a database of patients' names. 
Over the course of the study, researchers compare the names submitted by treatment programs to 
the names already in the database. This kind of study can be accomplished in any one of three 
ways: 








• Using patients' consents. All patients can sign consent forms that comply with §§2.31, 
permitting programs to disclose patient identifying information to researchers. The consent 
forms must specify the reason for the disclosure, what information will be disclosed to 
researchers, and an expiration date. (Note that this simple consent form does not permit the 
entity operating the database to disclose information to a treatment program or to anyone 
else.) 

• Using the research, audit, and evaluation exception. If the entity conducting the outcome 
monitoring satisfies the requirements of §§2.52 or 2.53, programs can disclose patient- 
identifying information for entry into and comparison with a database of patients' names. Again, 
no information from one treatment program can be disclosed to another. 

• Using the gualified service organization exception. The Federal confidentiality regulations permit 
AOD programs to enter into an agreement, called a qualified service organization agreement 
(QSOA), when they routinely need to share certain information with an outside agency that 
provides services to them. A QSOA is a written agreement between a program and a person or 
entity providing services to the program, in which that person or entity does the following 
(§§2.11, 2.12(c)(4)): 

1) Acknowledges that in receiving, storing, processing, or otherwise dealing with any 
patient records from a program, it is fully bound by [the Federal confidentiality] 
regulations 

2) Promises that, if necessary, it will resist injudicial proceedings any efforts to obtain 
access to patient records except as permitted by these regulations. 

An entity performing outcomes monitoring is providing a service to the program by studying 
how effective treatment is and how it might be improved. Disclosures made to an entity 
conducting outcomes monitoring with which programs have signed QSOAs must be limited to 
in formation that is needed to conduct the research. (See sample QSOA form in Exhibit 6-3.) 

• Using the internal communications exception. The Federal regulations permit programs to 
disclose information to an entity that has direct administrative control over them if that entity 
needs "the information in connection with [its] duties that arise out of the provision of 
diagnosis, treatment or referral for treatment of alcohol or drug abuse" (§§2.12(c)(3).). If the 
entity conducting the outcomes monitoring is the single State agency (SSA), the programs it runs 
could disclose the information it needs in order to conduct that monitoring under this 
exception, without patients' consent. Flowever, the SSA could not redisclose patient-identifying 
information to anyone else. 


Exhibit 6-3 

Qualified Service Organization Agreement 

XYZ Service Center ("the Center") and 
the 









(name of the program) 

("the Program") hereby enter into a qualified service organization agreement, whereby 
the Center agrees to provide the following services: 


(nature of services to be provided) 


Furthermore, the Center: 

1. Acknowledges that in receiving, storing, processing, or otherwise dealing with any 
information from the Program about the patients in the Program, the Center is fully 
bound by the provisions of the Federal regulations governing Confidentiality of Alcohol 
and Drug Abuse Patient Records, 42 CFR Part 2: and 

2. Undertakes to resist injudicial proceedings any effort to obtain access to information 
pertaining to patients otherwise than as expressly provided for in the Federal 
confidentiality regulations, 42 CFR Part 2. 

Executed this_day of_, 199_. 


President Program Director 

XYZ Service Center (name of program) 

(address) (address) 


Tracking patients in health, social welfare, and criminal justice systems. The Federal 
confidentiality regulations' restrictions on the disclosure and redisclosure of patient-identifying 
information generally make it difficult to track whether or not patients seek treatment from other 
medical or mental healthcare providers, apply for welfare benefits, or are arrested by law 
enforcement authorities. As noted above, the entity conducting outcome monitoring cannot 
simply turn over the names in its database to medical or welfare authorities or to officials of the 
criminal justice system, since the Federal confidentiality regulations prohibit this kind of 
disclosure. Flow can an entity conducting outcomes monitoring compare a database of patients' 
names with a similar database created by the healthcare, welfare, or criminal justice systems? 

A comparison of the names of patients in AOD programs with the names of patients receiving 
other medical or mental healthcare or welfare benefits can be accomplished in one of two ways: 

• If the entity conducting the outcomes monitoring is an 55/A that also administers programs and 
has jurisdiction over mental and mental healthcare and/or welfare. 














As noted above, the Federal regulations permit programs to disclose information to an entity that 
has direct administrative control over them if that entity needs "the information in connection 
with [its] duties that arise out of the provision of diagnosis, treatment, or referral for treatment of 
alcohol or drug abuse" (§ §2.12(c)(3)). If the entity conducting the outcomes monitoring is the 
SSA that runs AOD programs and also has jurisdiction over medical and mental healthcare, the 
AOD, medical, and mental healthcare providers under its jurisdiction could disclose the 
information the SSA needs in order to conduct outcomes monitoring under this exception. 
However, the SSA must resist the temptation to use this data for another purpose. For example, it 
could not redisclose the infonnation (names) it receives from AOD treatment providers to other 
AOD treatment providers or to mental healthcare providers or welfare authorities. 

• If the entity conducting the outcome monitoring is an SSA with jurisdiction solely over AOD 
treatment or is any other organization. 

Using patients' consents. All AOD patients can sign consent forms that comply with §§2.31, 
pennitting programs to disclose patient-identifying information to the entity conducting the 
research. The consent forms must specify the reason for the disclosure, what information will be 
disclosed to researchers, and an expiration date. Patients receiving care from other medical or 
mental healthcare providers can sign appropriate consent forms permitting those providers to 
disclose their names to the entity conducting the outcomes monitoring. No patient-identifying 
in formation that entity receives from AOD treatment programs can be redisclosed. 

Using the research, audit, and evaluation exception for AOD information and patients' 
consents for other information. If the entity conducting the outcomes monitoring satisfies the 
requirements of §§2.52 or 2.53, programs can disclose patient-identifying information for entry 
into and comparison with a database of patients’ names submitted by other medical and mental 
healthcare providers with patient consent. 

Using the qualified service organization exception for AOD information and patients' 
consents for other information. The entity conducting outcomes monitoring could collect 
in formation from AOD programs by using a QSOA, as outlined above, and from medical care, 
mental health, and welfare authorities by using appropriate patient consent. 

Tracking patients through the criminal justice system is particularly difficult. While an entity 
conducting outcomes monitoring can use any one of the mechanisms listed here to obtain the 
names of AOD patients, it may have difficulty obtaining information from the criminal justice 
system because of State laws protecting the confidentiality of that information. 


Other Issues 


Research and Mandatory Reporting Laws 



If a researcher uncovers child abuse or neglect while reviewing a patient's chart or interviewing 
the patient, must he or she make a report to State authorities? How can a report be made under 
the Federal confidentiality laws and regulations? 

Is the researcher required to report? All 50 States have statutes requiring reporting when there 
is reasonable cause to believe or suspect child abuse or neglect. While many State statutes are 
similar, each has different rules about what kinds of conditions must be reported, 9 who must 
report, and when and how reports must be made. Most States now require not only physicians 
but also educators and social service workers to report child abuse. What is unclear, however, is 
whether or not persons conducting research are mandated reporters. 10 This question is important 
because, as noted below, only those who are mandated by State law to report child abuse and 
neglect may do so under the Federal laws and regulations. 

Because of the variation in State laws, researchers who are concerned about this issue should 
consult an attorney familiar with State law to determine whether they are mandated to report 
child abuse. 

How can a researcher make a report? In 1986, the Federal confidentiality regulations were 
amended to permit AOD programs to comply with mandatory child abuse reporting laws. The 
Federal laws and regulations now permit compliance with State laws that require the reporting of 
child abuse and neglect. However, this exception to the general rule prohibiting disclosure of any 
information about a patient applies only to initial reports of child abuse or neglect. Programs and 
others cannot respond to followup requests for information or even subpoenas for additional 
information, even if the records are sought for use in civil or criminal proceedings resulting from 
the researcher's initial report, unless the patient consents or the appropriate court issues a special 
court order (see §§2.64 and 2.65). 

Any researcher contemplating making a child abuse report would be wise to check with the 
program to see if a report has already been made. 

Coding Patients' Identities 

If a researcher codes patients' names to protect their identities, can the intricate rules of the 
Federal confidentiality laws and regulations be disregarded? It depends. As noted above, the 
Federal regulations protect "patient-identifying information." Section 2.11 of the regulations 
defines this to mean the name, address, social security number, fingerprints, photograph, or 
similar information by which the identity of a patient can be detennined with reasonable 
accuracy and speed either directly or by reference to other publicly available information. The 
tenn does not include a number assigned to a patient by a program, if that number does not 
consist of or contain numbers (such as a social security or a driver's license number) that could 
be used to identify a patient with reasonable accuracy and speed from sources external to the 
program. 

Thus, if a research entity can truly code a patient's name so that the number created for each 
patient cannot be "used to identify a patient with reasonable accuracy and speed from sources 



external to the program," it need no longer be concerned with safeguarding infonnation about the 
patient. 


Conclusion 

Outcomes monitoring systems must be designed with patients' interests in mind. Since both 
in formed consent and issues of confidentiality can be central to research design, researchers 
should seek the advice of an attorney familiar with these issues to ensure that patients' rights are 
protected. 


Endnotes 

1. This chapter was written for the consensus panel by Margaret K. Brooks, Esq. The Center for 
Substance Abuse Treatment has published another document Confidentiality of Patient Records 
for Alcohol and Other Drug Treatment (Technical Assistance Publication Series 13) that 
addresses these issues. 

2. Hereinafter, citations in this section in the form "§§2..." refer to specific sections of 42 C.F.R., 
Part 2, implementing the Comprehensive Alcohol Abuse and Alcoholism Prevention, Treatment 
and Rehabilitation Act of 1970 (42 U.S.C. §§290dd-3) and the Drug Abuse Prevention, 
Treatment and Rehabilitation Act (42 U.S.C. §§290ee-3). 

3. Violating the regulations by disclosing information is punishable by a fine of up to $500 for a 
first offense or up to $5,000 for each subsequent offense (§§2.4). 

4. Two additional Federal laws permit the U.S. Attorney General and the Secretary of the U.S. 
Department of Health and Human Services (DHHS) to authorize researchers to withhold the 
names and identities of research subjects. The statute authorizing the Secretary of HHS to issue 
confidentiality certificates specifies that it applies to "persons engaged in biomedical, behavioral, 
clinical, or other research (including research on mental health, including research on the use and 
effect of alcohol and other psychoactive drugs)...." Once such authorization is issued, the 
researcher "may not be compelled in any Federal, State or local civil, criminal, administrative, 
legislative, or other proceeding to identify the subjects of research for which such authorization 
was obtained," (42 U.S.C. §§241 (d) permits the Secretary of HHS to issue confidentiality 
certificates; 21 U.S.C. §§872(c) permits the U.S. Attorney General to issue confidentiality 
certificates). 

These statutes may not be particularly useful for those conducting outcomes monitoring because 
most of the information they will gather will already be protected by the Federal confidentiality 
regulations. However, depending upon who gathers followup information from patients, it is 
possible that this information will not be protected by 42 C.F.R., Part 2, as it could be argued that 
the former patients are making voluntary disclosures of information about themselves directly to 


data gatherers. As with many questions about the design of outcomes monitoring, it is best to 
check with an attorney familiar with Federal and State laws and regulations for guidance. 


5. "Patient-identifying information means the name, address, social security number, 
fingerprints, photograph, or similar information by which the identity of a patient can be 
determined with reasonable accuracy and speed either directly or by reference to other publicly 
available information. The term does not include a number assigned to a patient by a program, if 
that number does not consist of or contain numbers (such as social security, or driver's license 
number) which could be used to identify a patient with reasonable accuracy and speed from 
sources external to the program" (§§2.11). 

6. Section 2.16 requires programs to keep written records in a secure room, a locked file cabinet, 
a safe, or other similar container. The program should have written procedures that regulate 
access to and use of patients' records. Either the program director or a single staff person should 
be designated to process inquiries and requests for infonnation. 

7. These particular entities may also copy or remove records, but only if they agree in writing to 
maintain patient-identifying information in accordance with the regulations' security 
requirements (see §§2.16), to destroy all patient-identifying information when the audit or 
evaluation is completed, and to redisclose patient information only 1) back to the program, 2) in 
accordance with a court order to investigate or prosecute the program (§§2.66), or 3) to a 
government agency overseeing a Medicare or Medicaid audit or evaluation (§§2.53(b). 

8. For a discussion of this issue in the context of a consent form permitting interviews with 
collateral sources, see the discussion on "Followup with collateral sources." 

9. For instance, in some States, certain professionals are required to report pregnant women who 
abuse drugs to State authorities. If such a requirement is part of the State's mandatory child abuse 
reporting law, the Federal regulations permit the report. If a separate statute mandates the report, 
the Federal regulations probably do not permit the report. 

10. Some State statutes are worded to require reports only from those persons who have face-to- 
face interaction with a child or adult who discloses abuse. 



Exhibit 6-3 

Qualified Service Organization Agreement 

XYZ Service Center ("the Center") and 
the 


(name of the program) 

("the Program") hereby enter into a qualified service organization agreement, whereby 
the Center agrees to provide the following services: 


(nature of services to be provided) 


Furthermore, the Center: 

1. Acknowledges that in receiving, storing, processing, or otherwise dealing with any 
information from the Program about the patients in the Program, the Center is fully 
bound by the provisions of the Federal regulations governing Confidentiality of Alcohol 
and Drug Abuse Patient Records, 42 CFR Part 2: and 

2. Undertakes to resist injudicial proceedings any effort to obtain access to information 
pertaining to patients otherwise than as expressly provided for in the Federal 
confidentiality regulations, 42 CFR Part 2. 

Executed this_day of_, 199_. 


President Program Director 

XYZ Service Center (name of program) 

(address) (address) 


Tracking patients in health, social welfare, and criminal justice systems. The Federal 
confidentiality regulations' restrictions on the disclosure and redisclosure of patient-identifying 
in formation generally make it difficult to track whether or not patients seek treatment from other 
medical or mental healthcare providers, apply for welfare benefits, or are arrested by law 
enforcement authorities. As noted above, the entity conducting outcome monitoring cannot 
simply turn over the names in its database to medical or welfare authorities or to officials of the 
criminal justice system, since the Federal confidentiality regulations prohibit this kind of 
















disclosure. How can an entity conducting outcomes monitoring compare a database of patients' 
names with a similar database created by the healthcare, welfare, or criminal justice systems? 

A comparison of the names of patients in AOD programs with the names of patients receiving 
other medical or mental healthcare or welfare benefits can be accomplished in one of two ways: 

• If the entity conducting the outcomes monitoring is an SS/\ that also administers programs and 
has jurisdiction over mental and mental healthcare and/or welfare. 

As noted above, the Federal regulations permit programs to disclose information to an entity that 
has direct administrative control over them if that entity needs "the information in connection 
with [its] duties that arise out of the provision of diagnosis, treatment, or referral for treatment of 
alcohol or drug abuse" (§§2.12(c)(3)). If the entity conducting the outcomes monitoring is the 
SSA that runs AOD programs and also has jurisdiction over medical and mental healthcare, the 
AOD, medical, and mental healthcare providers under its jurisdiction could disclose the 
information the SSA needs in order to conduct outcomes monitoring under this exception. 
However, the SSA must resist the temptation to use this data for another purpose. For example, it 
could not redisclose the infonnation (names) it receives from AOD treatment providers to other 
AOD treatment providers or to mental healthcare providers or welfare authorities. 

• If the entity conducting the outcome monitoring is an SS/\ with jurisdiction solely over AOD 
treatment or is any other organization. 

Using patients' consents. All AOD patients can sign consent forms that comply with §§2.31, 
pennitting programs to disclose patient-identifying information to the entity conducting the 
research. The consent forms must specify the reason for the disclosure, what information will be 
disclosed to researchers, and an expiration date. Patients receiving care from other medical or 
mental healthcare providers can sign appropriate consent fonns permitting those providers to 
disclose their names to the entity conducting the outcomes monitoring. No patient-identifying 
information that entity receives from AOD treatment programs can be redisclosed. 

Using the research, audit, and evaluation exception for AOD information and patients' 
consents for other information. If the entity conducting the outcomes monitoring satisfies the 
requirements of §§2.52 or 2.53, programs can disclose patient-identifying information for entry 
into and comparison with a database of patients’ names submitted by other medical and mental 
healthcare providers with patient consent. 

Using the qualified service organization exception for AOD information and patients' 
consents for other information. The entity conducting outcomes monitoring could collect 
information from AOD programs by using a QSOA, as outlined above, and from medical care, 
mental health, and welfare authorities by using appropriate patient consent. 

Tracking patients through the criminal justice system is particularly difficult. While an entity 
conducting outcomes monitoring can use any one of the mechanisms listed here to obtain the 
names of AOD patients, it may have difficulty obtaining infonnation from the criminal justice 
system because of State laws protecting the confidentiality of that infonnation. 




Other Issues 


Research and Mandatory Reporting Laws 

If a researcher uncovers child abuse or neglect while reviewing a patient's chart or interviewing 
the patient, must he or she make a report to State authorities? How can a report be made under 
the Federal confidentiality laws and regulations? 

Is the researcher required to report? All 50 States have statutes requiring reporting when there 
is reasonable cause to believe or suspect child abuse or neglect. While many State statutes are 
similar, each has different rules about what kinds of conditions must be reported, 9 who must 
report, and when and how reports must be made. Most States now require not only physicians 
but also educators and social service workers to report child abuse. What is unclear, however, is 
whether or not persons conducting research are mandated reporters. 10 This question is important 
because, as noted below, only those who are mandated by State law to report child abuse and 
neglect may do so under the Federal laws and regulations. 

Because of the variation in State laws, researchers who are concerned about this issue should 
consult an attorney familiar with State law to determine whether they are mandated to report 
child abuse. 

How can a researcher make a report? In 1986, the Federal confidentiality regulations were 
amended to permit AOD programs to comply with mandatory child abuse reporting laws. The 
Federal laws and regulations now permit compliance with State laws that require the reporting of 
child abuse and neglect. However, this exception to the general rule prohibiting disclosure of any 
in formation about a patient applies only to initial reports of child abuse or neglect. Programs and 
others cannot respond to followup requests for information or even subpoenas for additional 
information, even if the records are sought for use in civil or criminal proceedings resulting from 
the researcher's initial report, unless the patient consents or the appropriate court issues a special 
court order (see §§2.64 and 2.65). 

Any researcher contemplating making a child abuse report would be wise to check with the 
program to see if a report has already been made. 

Coding Patients' Identities 

If a researcher codes patients' names to protect their identities, can the intricate rules of the 
Federal confidentiality laws and regulations be disregarded? It depends. As noted above, the 
Federal regulations protect "patient-identifying information." Section 2.11 of the regulations 
defines this to mean the name, address, social security number, fingerprints, photograph, or 
similar information by which the identity of a patient can be detennined with reasonable 
accuracy and speed either directly or by reference to other publicly available information. The 
tenn does not include a number assigned to a patient by a program, if that number does not 
consist of or contain numbers (such as a social security or a driver's license number) that could 


be used to identify a patient with reasonable accuracy and speed from sources external to the 
program. 

Thus, if a research entity can truly code a patient's name so that the number created for each 
patient cannot be "used to identify a patient with reasonable accuracy and speed from sources 
external to the program," it need no longer be concerned with safeguarding infonnation about the 
patient. 


Conclusion 

Outcomes monitoring systems must be designed with patients' interests in mind. Since both 
in formed consent and issues of confidentiality can be central to research design, researchers 
should seek the advice of an attorney familiar with these issues to ensure that patients' rights are 
protected. 


Endnotes 

1. This chapter was written for the consensus panel by Margaret K. Brooks, Esq. The Center for 
Substance Abuse Treatment has published another document Confidentiality of Patient Records 
for Alcohol and Other Drug Treatment (Technical Assistance Publication Series 13) that 
addresses these issues. 

2. Hereinafter, citations in this section in the form "§§2..." refer to specific sections of 42 C.F.R., 
Part 2, implementing the Comprehensive Alcohol Abuse and Alcoholism Prevention, Treatment 
and Rehabilitation Act of 1970 (42 U.S.C. §§290dd-3) and the Drug Abuse Prevention, 
Treatment and Rehabilitation Act (42 U.S.C. §§290ee-3). 

3. Violating the regulations by disclosing information is punishable by a fine of up to $500 for a 
first offense or up to $5,000 for each subsequent offense (§§2.4). 

4. Two additional Federal laws permit the U.S. Attorney General and the Secretary of the U.S. 
Department of Health and Human Services (DHHS) to authorize researchers to withhold the 
names and identities of research subjects. The statute authorizing the Secretary of HHS to issue 
confidentiality certificates specifies that it applies to "persons engaged in biomedical, behavioral, 
clinical, or other research (including research on mental health, including research on the use and 
effect of alcohol and other psychoactive drugs)...." Once such authorization is issued, the 
researcher "may not be compelled in any Federal, State or local civil, criminal, administrative, 
legislative, or other proceeding to identify the subjects of research for which such authorization 
was obtained," (42 U.S.C. §§241 (d) permits the Secretary of HHS to issue confidentiality 
certificates; 21 U.S.C. §§872(c) permits the U.S. Attorney General to issue confidentiality 
certificates). 


These statutes may not be particularly useful for those conducting outcomes monitoring because 
most of the infonnation they will gather will already be protected by the Federal confidentiality 
regulations. However, depending upon who gathers followup information from patients, it is 
possible that this information will not be protected by 42 C.F.R., Part 2, as it could be argued that 
the former patients are making voluntary disclosures of information about themselves directly to 
data gatherers. As with many questions about the design of outcomes monitoring, it is best to 
check with an attorney familiar with Federal and State laws and regulations for guidance. 

5. "Patient-identifying information means the name, address, social security number, 
fingerprints, photograph, or similar information by which the identity of a patient can be 
determined with reasonable accuracy and speed either directly or by reference to other publicly 
available information. The term does not include a number assigned to a patient by a program, if 
that number does not consist of or contain numbers (such as social security, or driver's license 
number) which could be used to identify a patient with reasonable accuracy and speed from 
sources external to the program" (§§2.11). 

6. Section 2.16 requires programs to keep written records in a secure room, a locked file cabinet, 
a safe, or other similar container. The program should have written procedures that regulate 
access to and use of patients' records. Either the program director or a single staff person should 
be designated to process inquiries and requests for infonnation. 

7. These particular entities may also copy or remove records, but only if they agree in writing to 
maintain patient-identifying information in accordance with the regulations' security 
requirements (see §§2.16), to destroy all patient-identifying information when the audit or 
evaluation is completed, and to redisclose patient information only 1) back to the program, 2) in 
accordance with a court order to investigate or prosecute the program (§§2.66), or 3) to a 
government agency overseeing a Medicare or Medicaid audit or evaluation (§§2.53(b). 

8. For a discussion of this issue in the context of a consent form permitting interviews with 
collateral sources, see the discussion on "Followup with collateral sources." 

9. For instance, in some States, certain professionals are required to report pregnant women who 
abuse drugs to State authorities. If such a requirement is part of the State's mandatory child abuse 
reporting law, the Federal regulations permit the report. If a separate statute mandates the report, 
the Federal regulations probably do not permit the report. 


10. Some State statutes are worded to require reports only from those persons who have face-to- 
face interaction with a child or adult who discloses abuse. 



